Skip to main content

mlperf-viz

Visualize an MLPerf Endpoints submission folder — the same layout the submission checker expects — in a local browser dashboard.

pip install mlperf-viz
mlperf-viz /path/to/endpoints_results

That reads the submission tree, converts it to the dashboard's dataset, starts a local server, and opens the Pareto Explorer. No Node, no Bun, no account, and nothing leaves your machine.

Usage

mlperf-viz [PATH]

  PATH                 submission tree root (default: current directory)
  --version {1.0,0.7}  submission format to read (default: 1.0)
  --host HOST          interface to bind (default: 127.0.0.1)
  --port PORT          preferred port; the next free one is used if taken (default: 8000)
  --no-browser         do not open a browser window
  --export FILE        write the bundle JSON and exit
  --build DIR          write a self-contained copy of the site plus data to DIR and exit
  --strict             exit non-zero if any structural warning was raised
  -V, --cli-version    print this tool's version and exit

Note that --version selects the submission format, not this tool's version; -V prints the tool's version.

Expected layout

v1.0 (default)

The layout since endpoints_policies PR #119, anchored on a results/ tree whose leaves are per-concurrency point directories:

<Org>/<submission_id>/
    src/<implementation>/README.md
    docs/
    results/<System>/<Model>/r<N>/
        point.yaml
        result_summary.json
        accuracy_results.json
        system_desc.json
        config.yaml          (optional)

One <System>/<Model> pair is one Pareto curve, and one entry in the dashboard — §8.5 defines a result as "one system, one benchmark model, one dataset". A submission holding two models therefore shows as two entries.

system_desc.json sits in every point directory; §8.2 requires the points of a curve to agree on all of it but tps_utilization, so the first point describes the system and each point supplies its own utilisation.

v0.7 (--version 0.7)

The pre-PR#119 layout. Both variants are anchored on a systems/*.json file:

# Published results repository
<Org>/<System>/<Model>/
    systems/<System>.json
    points/point_<N>.yaml
    results/point_<N>/{run_metadata.json, results_summary.json, ...}

# Submission checker layout
<Org>/
    systems/<system_desc_id>.json
    pareto/<system_desc_id>/<model>/{points,results}/...
mlperf-viz ./endpoints_results_0.7 --version 0.7

PATH may be a whole results repository, a single organization directory, or one submission.

What changed between them

v0.7 v1.0
Point directory results/point_<N>/ results/<System>/<Model>/r<N>/
System description systems/<id>.json, one per model dir system_desc.json, one per point
Point config points/point_<N>.yaml point.yaml, inside the point
Metrics run_metadata.json result_summary.json
Accelerators flat on node_types[] nested in node_types[].accelerator_info[]
Model / dataset metadata on the system description on point.yaml
Accuracy on the system description accuracy_results.json

v0.7's run_metadata.json shipped system_tps, tps_per_user and qps already computed. v1.0 removed that file, so the visualizer derives them from the raw stat blocks using §9.1's definitions:

system_tps   = output_sequence_lengths.total / (duration_ns / 1e9)
tps_per_user = 1000 / tpot_p90_ms
qps          = n_samples_completed / (duration_ns / 1e9)

A v1.0 summary that still reports system_tps or tps_per_user is taken at its word. Timing blocks are nanoseconds, and percentile keys are read under both the "90" and "90.0" spellings, because endpoints and the checker's corpus disagree.

Missing or unreadable files produce warnings and are skipped, so a partial submission still renders. Use --strict in CI to turn those warnings into a non-zero exit. This tool does not validate a submission — use the submission checker for that.

Notes

  • Submissions are identified by their path — <Org>-<System>-<Model> in v0.7, <Org>-<submission_id>-<System>-<Model> in v1.0 — not by the submission_id field, which is frequently absent, empty, or shared between distinct submissions.
  • The format is chosen by the flag rather than sniffed from the tree: a directory that half-matches both layouts is a broken submission, and guessing would hide that. Reading a tree in the wrong format reports no submissions found, and names the flag.
  • A v1.0 node type that discloses more than one accelerator configuration is shown with the first; the dashboard displays one accelerator per node, and a warning says how many were omitted.
  • The dashboard loads web fonts and analytics from Google when online; offline it falls back to system fonts and those requests simply fail.

Development

The package embeds a prebuilt copy of the mlperf-viz dashboard. From a source checkout, build it first:

bash cli/scripts/build_webapp.sh   # requires Bun
cd cli && pip install -e '.[dev]' && pytest

License

Apache-2.0

Metadata

Release files for mlperf-viz 2.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mlperf-viz 2.0.0
File Size Uploaded Provenance Actions
mlperf_viz-2.0.0.tar.gz 458.0 kB With provenance Details for mlperf_viz-2.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for mlperf-viz 2.0.0
File Interpreter ABI Platform Actions
mlperf_viz-2.0.0-py3-none-any.whl Python 3 none any Details for mlperf_viz-2.0.0-py3-none-any.whl

Total release size: 968.0 kB

Release files / mlperf_viz-2.0.0.tar.gz

Download URL mlperf_viz-2.0.0.tar.gz
Size 458.0 kB
Tags Source
SHA-256 checksum
How to use checksums
3b8d89d1e3b22da529bcd3f467df8bc26e322abcc47d3baac26e90f7d71bde6c
BLAKE2b-256 checksum
How to use checksums
69ba9e4207f0a5ed1e8d4572b1eb9ec9a877851cc08a2cbdbe131f8198df6540
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 15, 2026.

Transparency log

Release files / mlperf_viz-2.0.0-py3-none-any.whl

Download URL mlperf_viz-2.0.0-py3-none-any.whl
Size 510.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c634aeda0d2eee153648c6b463a89c64507d0022fbe8abc11e332224ef5a85ff
BLAKE2b-256 checksum
How to use checksums
4eb14d0343213e9919e0310f0b3c1a8f847ac1dbaaf0bd9fcd94f4e0cb997929
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 15, 2026.

Transparency log

Security for mlperf-viz 2.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for mlperf_viz-2.0.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 15, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page