Experiment Tracker

A local-first experiment tracker. Metrics land in a JSONL file you own, stay queryable while the run is live, and can trigger alerts from an expression language. wandb and trackio are optional mirrors, not requirements.

📖 Documentation: https://hspk.github.io/expr_tracker/

import expr_tracker as et

et.init(project="demo", name="run-1", alert_rules=["zscore(loss[50]) > 3 => error: spike"])
for step in range(1000):
    et.log({"loss": loss, "lr": lr})
et.finish()

et.history(50)                    # the last 50 steps, as dicts
et.history(-1, output_type="pd")  # everything, as a DataFrame

Why

• The file is the source of truth. One JSON object per step, appended to metrics.jsonl. No server, no database, no vendor.
• History is queryable during the run. et.history(n) answers from an in-memory cache and touches the file only for what it evicted — 227 µs for history(50) whether the run has 1,000 steps or 100,000.
• Alerts are expressions, not callbacks. zscore(loss[50]) > 3 or isnan(loss) is parsed, validated, and evaluated against a rolling window. Rules can be replayed over a finished run to tune thresholds before you trust them.
• It stays out of the way. log() costs ~26 µs. A failed disk, a dead webhook or an unserialisable value degrades with a warning; none can stop training.

Install

uv add expr_tracker                 # local-first: click, loguru, pydantic only
uv add "expr_tracker[wandb]"        # mirror to Weights & Biases
uv add "expr_tracker[trackio]"      # mirror to trackio
uv add "expr_tracker[pandas]"       # history(output_type="pandas")
uv add "expr_tracker[all]"          # everything

Only the JSONL history is built in. A missing extra is reported with the exact install command; it never crashes a run.

Features

Examples

Every one runs offline, with no account and no network.

uv run python examples/quickstart.py
uv run python examples/alert_rules.py --fault spike
uv run python examples/multiprocess_pipeline.py --produce-ms 10 --train-ms 40

wandb compatibility

Migrating an existing script is usually one line:

# import wandb as et
import expr_tracker as et

init, log, finish, alert, log_artifact, use_artifact, Artifact, define_metric, run.summary, run.step, run.dir and run.url keep their wandb names and signatures. See the compatibility table.

Development

uv sync --all-extras
uv run pytest                                   # everything
uv run pytest -m "not slow and not benchmark"   # the fast suite
uv run pytest -m benchmark -s                   # timing and memory report
uv run pytest --cov=expr_tracker                # coverage
uv run ruff check src tests
uv run ruff format src tests

Test layout

Docs

uv run --group docs mkdocs serve    # preview at localhost:8000
uv run --group docs mkdocs build    # build into site/

Published to GitHub Pages by .github/workflows/docs.yaml on every push to main. Internals: docs/design.md (data model and key invariants) and docs/architecture.md (module map, read/write paths, concurrency model).

License

MIT