sparks

Training runs on your DGX Spark, with the curves next to the hardware they ran on.

Submit a training job from your laptop with one command. sparks builds your image, ships it to the box, queues the run and supervises it. Your loop reports its own loss and learning rate, and they land in Grafana beside the GPU power and temperature from the same minutes.

Before you run it

sparks provisions nothing. It needs a box already set up by sparkup: Prometheus with the remote-write receiver, Grafana, an image registry and the queue runner. On your own machine you need Docker and SSH to the box.

The distribution is sparks-dgx; the command and the import are both sparks.

Install

uv tool install sparks-dgx
sparks setup you@your-box

setup remembers the box, lets your Docker push to its registry, restarts Docker, and says when it is ready. The restart stops whatever containers you have running.

Submit a job

sparks submit --context ./examples --data ./examples/data \
  --name lora-r16 -- python /app/lora_finetune.py --epochs 12

--data arrives read-only at /data, also $SPARKS_DATA; read that path and never a laptop one. Name the script by absolute path, because the container's working directory is the box's shared directory rather than your image's.

Instrument the loop

from sparks.emit import track

with track(total=epochs * len(loader), tokens_per_step=batch_size * BLOCK) as run:
    for batch in loader:
        loss = train_one(batch)
        run.step(loss=float(loss))

run.step derives step, progress, eta_seconds and the rates. Off the box track reports nothing, so the same script runs on your laptop unguarded.

Watch it

sparks queue            # what is running and waiting
sparks logs <job>       # what it printed
sparks status <job>     # state, exit code, duration, energy
sparks wait <job>       # block until it ends

The README has the rest, and INSTALL_CLAUDE.md has the configuration and the traps.

MIT.