LaunchHelm

Training telemetry and live control for any Python or PyTorch loop — the Python SDK for the hosted LaunchHelm service at launchhelm.com.

Requires Python 3.11 or later on macOS, Linux, or Windows.

Install

pip install launchhelm

Quickstart

In Mission Control, open the organization's Projects page and choose New project — it creates the project and a project key in one step. The credential is shown once; save it before leaving:

export LAUNCHHELM_API_KEY=lh_...        # shown once — never commit it
export LAUNCHHELM_ORGANIZATION=...      # organization UUID
export LAUNCHHELM_PROJECT=...           # project UUID
# export LAUNCHHELM_URL=https://launchhelm.com   # optional; this is the default

A .env file in the working directory or a parent works too; exported shell variables take precedence. Then in the training script:

import launchhelm

with launchhelm.init("quickstart", config={"lr": 3e-4, "batch": 64}) as run:
    for step in range(100):
        loss = train_one_step()
        run.log({"loss": loss}, step=step)

The first argument is the project name, which must match the project the key belongs to; LAUNCHHELM_PROJECT is that project's UUID. With an organization-wide key, leave LAUNCHHELM_PROJECT unset and init reuses or creates the named project in the organization catalog.

Rich values

Beyond scalars, a stream can carry rendered values — text, media, tables, and histograms — via run.log with Text/Image/Audio/Video/Table/ Histogram objects or the matching run methods:

run.text("notes", "## epoch 4\nloss **0.25**", format="markdown")
run.image("frames", "confusion.png")            # background upload + spool
run.log_table("sweep", [("lr", "number")], [[3e-4]])
run.histogram("grad", gradient_samples, bins=64)

result = client.read_samples("stream-id", select={"mode": "latest", "limit": 16})

Native media uploads run off the training thread through the run's durable spool; flush() waits for the artifact to exist before the point is admitted. Sample reads return at most one point per axis value — the newest write wins — so a text stream keeps one value per step, not an appended log.

PyTorch

launchhelm.torch is an optional adapter (pip install "launchhelm[torch]") for arbitrary training loops, including DDP. import launchhelm itself never imports torch.

import launchhelm.torch as lht

tr = lht.init("urm-llm", config={"lr": 3e-4, "batch": 64})
model, optimizer = build()
tr.watch(model, optimizer, scheduler=sched)

for step in range(steps):
    loss = train_one_step()
    tr.clip_grad_norm_()          # current (controllable) max norm
    optimizer.step(); sched.step()
    tr.step(loss=loss, samples=batch_size)   # logs + services control; call on every rank

tr.save_checkpoint(extra={"epoch": epoch})
tr.finish()

tr.step logs the built-in series (train/loss, optim/lr/group<i>, optim/grad_norm, perf/step_time_s, perf/samples_per_s, perf/tokens_per_s, optim/loss_scale with a watched GradScaler, cuda/max_memory_allocated_bytes on CUDA) and services the lht safe point: commands issued from the UI apply only inside step, never between backward and optimizer.step. Registered controls: lr (scale or absolute, scheduler-aware), weight_decay, grad_clip, dropout, checkpoint, eval when eval_fn is passed to watch, and tr.tunable(name, ...) for custom typed knobs.

Distributed: call lht.init on every rank — rank 0 opens the run and every rank is a rank-<r> participant. step is a collective; a control applied at a step applies on every rank at that step, keeping DDP replicas identical, and a rank that fails the apply raises DivergedStateError on all ranks after the command is journaled as outcome_unknown.

save_checkpoint writes a torch.load(weights_only=True)-safe payload — model (DDP-unwrapped), optimizer, scheduler, scaler, step, per-rank RNG streams, tunables, control state, extra — uploads it, and keeps it so a crash-resume finds the latest. load_checkpoint("latest") restores model/optimizer state identically on every rank plus each rank's own RNG stream, and returns extra. Honest limits: GPU/NCCL/multi-node operation is not yet qualified; FSDP/sharded models are refused with CheckpointError (configure a full state dict first); NumPy's global RNG requires legacy MT19937 state; sampler/data-iterator positions and dataloader-worker state are the caller's responsibility — restoring weights and global RNG alone does not faithfully reconstruct a shuffled or multi-worker data pipeline.

System metrics without torch: launchhelm.system.start(run, interval=15.0) (launchhelm[system]) emits CPU, RSS, disk/network rates (psutil) and per-GPU stats (NVML) under system/... on a daemon thread that never blocks logging.

Framework callbacks: launchhelm.torch.hf.LaunchHelmCallback (launchhelm[hf] extra) and launchhelm.torch.lightning.LaunchHelmCallback (launchhelm[lightning] extra) wire the same init/log/control/checkpoint/finish lifecycle through each framework's callback API.

Delivery guarantees

Memory and spool limits are serialized-payload budgets; SQLite, journal and allocator overhead are separate. Command tombstones are retained under a separate bounded journal budget; exhaustion fails closed. Unknown effects are never automatically re-executed. On Windows the local spool's file permissions come from the user profile's ACL rather than POSIX modes.

Links

• Documentation: launchhelm.com/docs — getting started, Python guide, concepts
• Changelog: CHANGELOG.md, shipped in the source distribution