momentum-cli

The Momentum customer CLI + experiment-logging SDK: authenticate and bulk-upload field data straight to your workspace's storage bucket, and report training/eval runs from your own compute into your workspace's experiment tracker.

PyPI distribution: ydderd-momentum-cli · Homebrew formula: momentum-cli · command: momentum. (The clean momentum-cli PyPI name was taken, so the distribution carries the ydderd- prefix; the import package momentum_cli, the momentum command, and the brew name are unaffected.)

Why this is a separate package

The CLI talks to the Momentum API purely over HTTP (and to R2 over S3). It shares no Python code with the backend, so it ships with a tiny dependency set — httpx + boto3 — instead of the full server stack (torch, opencv, fastapi, …). That keeps the install small and avoids shipping the backend's AGPL detector to customers.

Install

brew install ydderd/momentum/momentum-cli
# or:
pipx install ydderd-momentum-cli
momentum --help

Usage

momentum auth login                       # opens a browser for workspace approval
momentum auth whoami                      # confirm tenant
momentum upload ./your-data --scan        # bulk upload + trigger ingest
momentum datasets list                    # list datasets (id, name, kind) to pick a target
momentum datasets assign <session_id> --dataset "Evaluation rollouts"  # assign an upload to a Dataset
momentum ingest status                    # ingest ledger stats
momentum eval submit --model hf://lab/pi05-fast --benchmark bench_roboarena@3  # open an eval run, print URL
momentum trials log --policy hf://lab/pi05-fast --n 20 --successes 14 --calibration-set pcsk-…
momentum trials log --csv trials.csv      # bulk floor tallies (per-row partial success)
momentum secrets set lab-bucket           # store a secret (value from stdin); prints its creds_ref
momentum secrets list                     # secret names + configured (never values)

upload always writes to your workspace's one raw prefix — there's no target to choose. Whether what you uploaded is raw drone video (needs extraction) or already-extracted frames is classified server-side once it lands, not by the client beforehand.

Uploaded files are reported to the platform in batches of 200. An interrupted run resumes with momentum upload <dir> --resume <session_id>; up to 200 files at the tail are re-checked, and a file already in place is skipped rather than sent again.

For headless/CI use, skip the browser with a token minted by any provisioned workspace user: momentum auth login --token <fw_cli_…>.

If browser approval fails or times out, the CLI exits with a retry message instead of a traceback. Run momentum auth login again; if you switched workspaces in the browser, refresh the app and approve from the target workspace.

Config is stored at ~/.momentum/config.json (an existing ~/.flywheel/config.json is copied over once on first use). Auth precedence: MOMENTUM_CLI_TOKEN env > config file.

Experiment-logging SDK

Training and eval runs executed on your own compute (Modal, Brev, a lab box) report themselves into your workspace's experiment tracker — W&B-style, and safe to leave in production training code (a logging failure never raises into the train):

import momentum_cli as momentum

run = momentum.init(name="my_sft_run", tags=["sft"], config={"iters": 800, "lr": 2e-4},
                    provider="modal")
run.log({"train/loss": 0.42}, step=100)
run.finish(status="succeeded", checkpoint_ref="r2://bucket/ckpt")

# later — scoring results and billed cost arrive after the train, so annotation
# works on finished runs:
momentum.annotate(run.id, results={"auroc": {"value": 0.61, "ci": [0.55, 0.67]}})

Eval runs (policy context — the CI-integration path)

An eval process (a lab rig, Modal, the robot) evaluates model × benchmark@version and streams its rollouts back. Same never-raise/heartbeat/reattach posture as training runs; rollouts buffer and flush in batches, each with a client-generated id so a re-sent batch is idempotent:

ev = momentum.eval_run(benchmark="bench_roboarena@3", model="hf://lab/pi05-fast", seeds=3)
ev.log_rollout(scenario="scn_pick", seed=0, status="success",
               scorer={"success": True, "task_progress": 1.0}, latency_p50=61.0)
ev.log_rollout(scenario="scn_pick", seed=1, status="fail", scorer={"success": False})
ev.finish()                                   # flushes any buffered rollouts first
ev.annotate(results={"headline": {"value": 0.5, "ci": [0.31, 0.69]}})   # post-hoc scoring

eval_run() prints the run URL on create; eval_run(run_id=…) (or MOMENTUM_EVAL_RUN_ID) reattaches after a preemption. Runs land in the UI under Eval runs.

Real trials (floor tallies → calibration audit)

Report real-robot trials of a policy; landing trials that ground a calibration set recomputes that world model's τ/ρ trust:

momentum.real_trials.log(policy="hf://lab/pi05-fast", scenario="scn_pick",
                         n=20, successes=14, operator="alice", calibration_set="pcsk-…")

report = momentum.real_trials.log_csv("trials.csv")   # a path or raw CSV text; per-row partial success
print(report["accepted"], report["rejected"])

Episode ingestion and secrets

Episode creation uses the same upload-session workflow as every other ingest path. Upload the source dataset unchanged, then assign the returned session to its Dataset. Normalization creates database-owned Episode identities and the application-readable artifacts:

momentum upload ./session_042
momentum datasets assign <session_id> --dataset "Training demonstrations"

The Episode SDK is read/curate-only. Named secrets remain available for supported external-service configuration; values are encrypted at rest and never returned by a read.

Auth: MOMENTUM_API_KEY env (a fw_cli_… token — inject it as a secret in your training environment), falling back to the token saved by momentum auth login. MOMENTUM_API_URL overrides the API endpoint. with momentum.init(...) as run: (and momentum.eval_run(...)) marks the run failed (with the exception) if the block raises. Runs land in the workspace UI under Experiments / Eval runs.

Release/consumption mechanics (PyPI, git-ref installs, versioning): see PUBLISHING.md.

Developer notes

These knobs exist for Momentum developers and are intentionally hidden from customer-facing help and docs:

• --api-url <url> on momentum auth login — persist a non-production API base URL to the config (e.g. a local API). Hidden via argparse.SUPPRESS.
• MOMENTUM_API_URL env — override the API base per-invocation. Takes precedence over the config file.

Precedence for the API base URL: MOMENTUM_API_URL env > api_url in config > default (https://api-aws.momentumbots.io/api, the hosted production API on AWS).

Point the CLI at a local backend during development:

MOMENTUM_API_URL=http://localhost:8000 momentum auth whoami
# or persist it:
momentum auth login --token <fw_cli_…> --api-url http://localhost:8000

Local development

cd cli
uv sync
uv run momentum --help
uv run pytest

Releasing (PyPI + Homebrew)

PyPI is the source of truth; the Homebrew formula wraps the published PyPI sdist.

1. Publish to PyPI — via GitHub Actions (Trusted Publishing, no token)

The .github/workflows/publish-cli.yml workflow builds and publishes over OIDC. Cut a release by pushing a namespaced tag from the monorepo default branch:

git tag cli-v0.1.0 && git push origin cli-v0.1.0

The PyPI project is ydderd-momentum-cli, published from ydderd/momentum via the pypi environment. (First publish activates the "pending" Trusted Publisher and creates the project.)

2. Update the Homebrew tap formula

After the PyPI release exists, point release.sh at your tap checkout — with SKIP_PUBLISH=1 it skips the upload, fetches the published sdist's url/sha256, writes an explicit formula version, refuses placeholder formula values, and regenerates Python resource blocks:

SKIP_PUBLISH=1 \
FORMULA_PATH=/path/to/homebrew-momentum/Formula/momentum-cli.rb \
  cli/scripts/release.sh

Then commit + push the tap. Customers install with:

brew install ydderd/momentum/momentum-cli

release.sh can also publish to PyPI itself (UV_PUBLISH_TOKEN=pypi-… cli/scripts/release.sh) if you prefer a token-based local release over the GitHub Action.

Bumping a release: change version in pyproject.toml and src/momentum_cli/__init__.py, push a new cli-v* tag, then re-run step 2. release.sh refuses to continue if those versions drift.