benchloop-client (bl, bmcli)
Command line tool for fetching and downloading benchmark analysis data
(run metrics, trajectory, pre-analysis reports, task/job lists, and artifacts)
by group_id / job_id.
PyPI package name is
benchloop_client; the installed commands areblandbmcli.
Install package
pip install benchloop_client
bl --api-key $BENCH_LOOP_API_KEY overview get -j $JOB_ID
bl --api-key $BENCH_LOOP_API_KEY metrics get -j $JOB_ID
# Option 2: config ~/.benchloop/config.toml by cli, and run bl without --api-key option:
bl config set --api-key $BENCHLOOP_API_KEY --host https://benchmark-analysis.alibaba-inc.com
bl overview get -j $JOB_ID
bl metrics get -j $JOB_ID
Install and run from source
# with uv (recommended)
uv sync
uv run bl --help
# or editable install with pip
python -m venv .venv && source .venv/bin/activate
pip install -e .
bl --help
Auth & config
Authentication uses a personal api key sent as the X-API-Key header.
Precedence: CLI flag > env var > ~/.benchloop/config.toml > default.
- Env:
BENCHLOOP_API_KEY(orBENCHMARK_API_KEY),BENCHLOOP_HOST,BENCHLOOP_OUTPUT_DIR - Default host (prod):
https://benchmark-analysis.alibaba-inc.com - Pre host:
https://pre-benchmark-analysis.alibaba-inc.com
bl config set --api-key $BENCHLOOP_API_KEY --host https://benchmark-analysis.alibaba-inc.com
Usage
Two-segment bl <resource> <action> form. get prints a JSON array to
stdout (pipeable to jq); download writes the standard file layout.
bl metrics get -j J123 | jq '.[0]'
bl metrics download -g G1 --scope all -o ./out
bl trajectory download -g G1,G2 -o ./out
bl trajectory download --task-id TASK_ID -o ./out
bl report download -j J123 -o ./out
bl report get -g G123
bl report download -g G123 -o ./out
bl jobs get -g G1 --all # get supports --all (auto-paginate)
bl jobs download -g G1 # download is always full
bl artifacts download -g G1 -o ./out
bl pull -g G1 --with metrics,trajectory,jobs,artifacts -o ./out
IDs accept repeated flags, comma-separated values, and - to read from stdin.
-g and -j are mutually exclusive in a single command. For trajectory
downloads, --task-id is a third mutually exclusive input mode.
Download filtered trajectories by task ID
The benchmark analysis Jobs page can turn the current filters into a trajectory download task. Copy the command shown by the page and run it locally:
bl trajectory download --task-id TASK_ID -o ./out
The CLI fetches the task's complete job ID list, downloads job trajectories with
fixed concurrency, and prints count/ETA progress to stderr. It creates
./out/TASK_ID_trajectories.zip, containing one <jobId>_trajectory.json per
successful job and a manifest.json with total, success, and failure details.
If only some jobs fail, the successful files are still archived and the command
exits with code 1. If every job fails, no empty ZIP is created.
Exit codes
| code | meaning |
|---|---|
| 0 | all success |
| 1 | partial failure (some ids failed) |
| 2 | usage / parameter error |
| 3 | auth failure (HTTP 401) |
| 4 | network / timeout |
stdout carries data only; logs and progress go to stderr. --quiet suppresses
non-error logs, --verbose adds debug logs.
Release new package
uv build
twine check dist/benchloop_client-0.1.7-py3-none-any.whl
twine upload dist/benchloop_client-0.1.7-py3-none-any.whl --verbose
Metadata
Release files for benchloop-client 0.2.14
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| benchloop_client-0.2.14-py3-none-any.whl | Python 3 | none | any | Details for benchloop_client-0.2.14-py3-none-any.whl |
Release files / benchloop_client-0.2.14-py3-none-any.whl
| Download URL | benchloop_client-0.2.14-py3-none-any.whl |
|---|---|
| Size | 36.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a82ea6c6bf412829c5249886ebd13982a8613b2abde9180b00e5bca8dfa0767f
|
|
BLAKE2b-256 checksum How to use checksums |
2e35591b338e880dc20df60c761ed1226c7a4687ef02dbc38e53df0c9ceb9a75
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.8
|
Security for benchloop-client 0.2.14
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.