benchpress-sdk 0.1.0

Emit structured benchmark records from your training and experiment scripts.

benchpress-sdk

Emit structured benchmark records from your training and experiment scripts.

benchpress-sdk is a tiny, dependency-free logging shim. You call a few functions — start_run, log_metrics, log_device, log_artifact, end_run — and it writes structured records to stdout, one per line, in a simple [BENCH:<KIND>] <json> format. A runner that launches your script reads those lines back. Because everything crosses the process boundary as text on stdout, your script needs nothing from the runner and stays decoupled from it.

• Zero required dependencies — pure standard library at import time.
• Python 3.6+ — works in older training environments.
• Framework-agnostic — metric values from torch / numpy / plain floats are coerced for you.

Install

pip install benchpress-sdk

import benchpress

The distribution is named benchpress-sdk; you import it as benchpress.

Quickstart

import benchpress

params = benchpress.start_run(run_name="demo", params={"lr": 0.01, "epochs": 3})
benchpress.log_device("cpu")

for step in range(params["epochs"]):
    loss = 1.0 / (step + 1)
    benchpress.log_metrics({"loss": loss}, step=step)

benchpress.end_run("FINISHED")

Run it and the records appear on stdout:

[BENCH:START]   {"run_id": "...", "run_name": "demo", ...}
[BENCH:DEVICE]  {"device": "cpu", ...}
[BENCH:METRICS] {"step": 0, "metrics": {"loss": 1.0}, ...}
...
[BENCH:END]     {"status": "success", "duration": ...}

Declaring inputs

A runner can inject --key=value flags. Declare each one inline, next to the code that uses it:

from benchpress.sdk import DeclareArg

data_path = DeclareArg("data_path", str, "./data.csv", "Path to the input file")

Run standalone and the default is used; run under a runner and the injected value is picked up.

Saving synthetic arrays (optional)

save_synthetic writes an array as .npy plus a flat .csv. It needs numpy + pandas, kept as an optional extra so the base install stays dependency-free:

pip install "benchpress-sdk[sdk]"

The wire format

One record per line on stdout:

[BENCH:<KIND>] <single-line-json>

<KIND> is one of START, DEVICE, METRICS, ARTIFACT, TAG, END. Lines that don't match are treated as ordinary log output and passed through untouched. Each record is flushed immediately, so a reader sees it in real time — even across a pipe into WSL or a container.

License

Apache-2.0 — see LICENSE and NOTICE.

Learn more at benchpress.run.