Experiment Tracker

A local-first experiment tracker. Metrics land in a JSONL file you own, stay queryable while the run is live, and can trigger alerts from an expression language. wandb and trackio are optional mirrors, not requirements.

📖 Documentation: https://hspk.github.io/expr_tracker/

import expr_tracker as et

et.init(project="demo", name="run-1", alert_rules=["zscore(loss[50]) > 3 => error: spike"])
for step in range(1000):
    et.log({"loss": loss, "lr": lr})
et.finish()

et.history(50)                    # the last 50 steps, as dicts
et.history(-1, output_type="pd")  # everything, as a DataFrame

Why

• The file is the source of truth. One JSON object per step, appended to metrics.jsonl, automatically split into retained segments at 100 MiB. History queries read all segments transparently. No server, no database, no vendor.

• History is queryable during the run. et.history(n) answers from an in-memory cache and reads older records from the local files.

• Alerts are expressions, not callbacks. zscore(loss[50]) > 3 or isnan(loss) is parsed, validated, and evaluated against a rolling window. Rules can be replayed over a finished run to tune thresholds before you trust them.

• It stays out of the way. A failed disk, a dead webhook or an unserialisable value degrades with a warning; none can stop training.

Local Web workspace

The native development build includes a local, read-only Web workspace:

et serve --root ./tracker/jsonl

Open http://127.0.0.1:9110 to inspect metrics, compare runs, follow new records, and view config, summary, and spans across rotation segments. Enter the random key printed at startup in the viewer's login form. The default listener is 0.0.0.0; use a trusted network or HTTPS/SSH for remote access. Use Add run to select each run's directory on the server. Added runs are remembered locally across reloads; subdirectories are not scanned for runs. See the viewer guide and native build instructions.

Install

uv add expr_tracker                 # local-first; no remote backend required
uv add "expr_tracker[wandb]"        # mirror to Weights & Biases
uv add "expr_tracker[trackio]"      # mirror to trackio
uv add "expr_tracker[pandas]"       # history(output_type="pandas")
uv add "expr_tracker[all]"          # everything

Only the JSONL history is built in. A missing extra is reported with the exact install command; it never crashes a run.

Features

Examples

Every one runs offline, with no account and no network.

uv run python examples/quickstart.py
uv run python examples/alert_rules.py --fault spike
uv run python examples/multiprocess_pipeline.py --produce-ms 10 --train-ms 40

wandb compatibility

Migrating an existing script is usually one line:

# import wandb as et
import expr_tracker as et

init, log, finish, alert, log_artifact, use_artifact, Artifact, define_metric, run.summary, run.step, run.dir and run.url keep their wandb names and signatures. See the compatibility table.

Development

uv sync --all-extras
uv run pytest                                   # everything
uv run pytest -m "not slow and not benchmark"   # the fast suite
uv run pytest -m benchmark -s                   # timing and memory report
uv run pytest --cov=expr_tracker                # coverage
uv run ruff check src tests
uv run ruff format src tests

Test layout

Docs

uv run --group docs mkdocs serve    # preview at localhost:8000
uv run --group docs mkdocs build    # build into site/

Published to GitHub Pages by .github/workflows/docs.yaml on every push to main. Internals: docs/design.md (data model and key invariants) and docs/architecture.md (module map, read/write paths, concurrency model).

License

MIT