5 min read

Attach Logs and Screenshots to a SARA Chat About Your Cloud GPU VM

Skyportal
Attach Logs and Screenshots to a SARA Chat About Your Cloud GPU VM

Say your training job runs on a GPU VM on AWS, and a GitHub Actions job builds the container image it runs in. When a run fails, the clues end up in different places. The VM has its own logs, but the build log is in CI, a teammate sends you a screenshot of the error, and the log from yesterday's run is a file you saved before the spot instance it ran on was reclaimed.

Skyportal's chat takes those files as attachments. Attach a log, a CSV, a JSON or YAML config, or a screenshot to your message, and SARA, the agent in Skyportal's chat, reads it while it checks the VM you selected. Nothing is copied to the VM, and Skyportal doesn't need your AWS keys for any of this, only the SSH login it already has for the VM.

What you can attach

In the web app, click the paperclip next to the message box, or drag the files onto the box. Each file shows up with its size and a button to remove it before you send. One message takes up to five files, 10 MB each and 20 MB in total. When you send, the files upload first and the message goes out right after them.

Images can be PNG, JPEG, GIF or WebP, up to 4096 pixels on each side. Everything else has to be plain text: logs, CSV, JSON, YAML, .out and .err files, rotated logs such as vllm.log.1, and files like syslog or dmesg. Skyportal checks that a text file really is text (UTF-8, or UTF-16 with a byte order mark), so PDFs and zip files are refused.

What SARA reads from a text file

Skyportal masks the secrets it recognizes in a text file: private keys, API keys, tokens, passwords, passwords inside connection strings and URLs, and kubeconfig credentials. The copy it keeps is the masked one, and that's the text SARA reads.

SARA doesn't read the whole file, though. In one chat it gets up to about 3,000 characters of attached text, and from a long file it takes the end, which is where a traceback usually is. You can attach a 10 MB log, but only its last lines reach SARA. If the error you care about is in the middle, copy that part into a smaller file and attach that one.

Attachments are read in the order you added them, so a long log attached earlier in the chat can use up all of that space. For a file about something else, start a new chat.

What SARA gets from a screenshot

SARA doesn't look at the image itself. Skyportal asks an OpenAI model to describe it, including any error text it can read, and SARA works from that description. Crop the screenshot to the error, so the text in it is big enough to read.

Images aren't scanned for secrets the way text files are. Crop out tokens, keys and anything else you wouldn't paste into the chat.

Ask about the file and the VM together

Select the VM in the chat and mention the file in your question, by name or as "the attached log". SARA reads attachments when a question points at them, then checks the VM with ordinary commands:

  • "The attached log is from yesterday's run on a spot instance that's gone now. It ran out of GPU memory. How much does this VM have free?"
  • "Here's the CI log for the image this VM runs. Does the CUDA version it was built with match the driver here?"
  • "My teammate sent this screenshot of the error. Does this box run the PyTorch version it mentions?"

SARA answers with commands such as nvidia-smi, pip show or cat, and each one follows your approval settings. Skyportal processes a file in the background after the upload, so if SARA says a file isn't ready yet, ask again a moment later.

If the log is already on the VM, skip the attachment and ask SARA to read it there. On the VM it can grep the whole file, while an attachment only gives it the end.

From the terminal

The Skyportal CLI attaches files too. It needs Python 3.11 or newer, and running skyportalai with no arguments opens its interactive shell:

python -m pip install skyportalai
skyportalai

Inside it, log in once, pick the VM and send your first question. Files attach to a chat that has already started, so /upload comes after that:

/login
/server my-gpu-box
why did the last training run on this box stop?
/upload ./ci-build.log ./train-tail.log

The CLI prints each file it attached with its size. Then ask about the file by name, the same way you would in the web app. You can also drop a file onto the terminal window: the terminal types its full path, and the CLI uploads the file instead of sending the path as a message. Empty files and files over 10 MB are refused before anything is uploaded. When you move on to an unrelated file, /new starts a fresh chat.

Get started

The Free plan is enough to try this. Sign up, add your VM over SSH, and attach the next log or screenshot you'd send a teammate. The /upload command is on the interactive terminal page of the docs.