GPU capacity moves around. This month your training box might be on Lambda, the inference endpoint on an EC2 GPU instance, and a test VM on RunPod or Vast.ai, because that's where the cards were available at a price you could live with. Each one is billed by the hour, and each one has its own console.
Skyportal doesn't need access to any of those cloud accounts. It needs an SSH login to the VM. You add the host once, and from then on you get GPU and host monitoring, your Weights & Biases or MLflow runs, and SARA, the AI agent in Skyportal's chat, working on that VM with you. VMs from different clouds end up in the same Hosts list.
Adding the VM
In the app, open Hosts, click New Host and stay on the SSH host tab. You fill in:
- The login, as
username@IP. On EC2 that'subuntu@plus the public IP for Ubuntu images, orec2-user@for Amazon Linux. - A password or a private key. Paste the key, and if it has a passphrase, there's a field for that.
- A name for the host and an environment: Production, Staging, Dev or Custom. Put the VM that serves your users under Production, where SARA can only run read-only commands.
Test Connection does a real SSH login with a 10 second timeout. Skyportal logs in again when you click Save, and only stores the host if that works, so a wrong username or a closed port fails right there instead of later in a chat.
Skyportal connects from its own servers, so the VM's SSH port has to accept connections from the internet. On AWS that's an inbound rule for port 22 in the instance's security group, on GCP a firewall rule, and on Azure a network security group rule. Use key authentication.
The New Host form assumes port 22. RunPod and Vast.ai usually hand you a different SSH port, so add those VMs from the chat instead: tell SARA you want to add a server and it asks for the same details one at a time, including the port. Credentials you paste there are masked and never sent to the model.
What you get without asking
Every 30 seconds, Skyportal logs in over SSH and reads the basics: CPU, memory, disk, load, and GPU utilization and memory from nvidia-smi. It also records the GPU model, the CUDA version, the OS, kernel, Python and Docker versions, failed systemd services, OOM kills, and journal warnings and errors from the last 24 hours.
The Monitor Dashboard shows this per host. Each GPU gets a card with the processes using it, and there are history charts for GPU memory and utilization, CPU and RAM, plus disks, network and the latest journalctl warnings and errors.
When you add the host from the New Host form, Skyportal also installs a small agent on the VM. It lives in ~/.skyportalai with its own Python venv, runs every minute from cron or a systemd timer, and picks up the Weights & Biases and MLflow runs it finds on the host, so your loss curves and run metrics show up in Observability. Hosts added from the chat don't get the agent yet.
When the public IP changes
Stop and start an EC2 instance that has no Elastic IP, or a GCP VM with an ephemeral external IP, and it comes back with a new public address. Skyportal still has the old one, so the host stops reporting until you click Edit Server on it and put in the new IP. If you stop and start a VM often, give it a static address (an Elastic IP on AWS, a reserved static IP on GCP) and you won't have to think about it.
Asking SARA about the VM
Select the host in the chat and ask in plain words, the way you'd ask a teammate who has a shell open on the box:
- "Has the GPU on this box done anything this week, or am I paying for an idle instance?"
- "GPU memory is at 95% but my job finished. What's still holding it?"
- "The model server on this host died around 3am. What happened?"
SARA starts with what Skyportal already stored. It can pull trends for up to 7 days, or the sample closest to a given time, which covers both the "this week" and the "around 3am" questions. When it needs more, it runs commands on the host. You see each command before it runs, and it waits for your approval unless you've already allowed that command.
Because every VM sits in the same list, you can put the Lambda box and the EC2 instance in one chat and ask once, for example whether both run the same CUDA version.
The same works from the terminal:
pip install skyportalai
skyportalaiInside the CLI, run /login once, list your hosts with /servers, pick the VM with /server, and ask. /upload attaches a training log, a config, or a CSV to the conversation.
/login
/servers
/server my-gpu-boxNo cloud credentials
Skyportal never asks for AWS, GCP or Azure keys to do any of this. It doesn't see your cloud bill and can't start or stop instances through your cloud account. Everything goes through the SSH login you gave it, which makes it easy to try on a VM you already have running.
If your workload runs on EKS, GKE or AKS instead, the Kubernetes cluster tab in New Host takes a kubeconfig. The kubeconfigs those clouds hand you use an exec plugin, which Skyportal doesn't run, so create a service account token with kubectl create token and use the Enter details option, which asks for the API server URL and the token. Those tokens expire, after an hour by default, so ask for a longer one with --duration. The API server has to be reachable from the internet, which is how EKS, GKE and AKS endpoints are set up unless you made them private. The Kubernetes guide lists what's accepted.
Get started
The Free plan is enough to try this on one VM. Sign up, add the host, and ask SARA something you'd normally SSH in to check yourself. The CLI needs Python 3.11+ and is on PyPI.