szl-wave1-report
Control before action. Evidence after.
Product · Proof · GitHub organization
The Wave 1 capstone. Four SZL harnesses emit hash-chained receipts:
szl-retrieval-bench— retrieval quality (nDCG / Recall / MRR / MAP)szl-engine-bench— serving throughput and latency (TTFT, tok/s)szl-quant-bench— quantization quality curves (cosine / KL / top-1)szl-calibration— receipted calibration gates
This package verifies every chain, then consolidates the results into one
report with a single master receipt hash — the chain-of-chains. If any
source chain is broken, the whole report is INVALID. No partial credit.
Version 0.2.1 reads the actual stdlib self_hash/run records, calibration's
hash/payload records (with its distinct genesis), and legacy flat
chain_hash records. It verifies each format with its native canonicalization;
it never rewrites or re-signs source records to make them pass.
Doctrine
- Fail closed. One tampered or broken chain voids the report; the reason names the harness.
- Only
MEASUREDenters the table.BLOCKEDlanes are listed with their reasons, never interpolated. - No cross-harness ranking. A tok/s number and an nDCG number never appear in the same ordered list.
- Order-independent. The master hash is computed over sorted harness names — same chains, same hash, any machine.
Usage
pip install -e . pytest
python -m pytest tests/ -q
python -m szl_wave1_report --chain szl-retrieval-bench=retrieval.json --chain szl-calibration=calibration.jsonl --format bundle --output wave.zip
The CLI accepts JSON receipt lists, JSON objects containing a chain list, or
JSONL chains. Destination files must be new. Invalid chains produce an INVALID
report and a nonzero exit; malformed files fail before a report is produced.
The JSON includes the full terminal hashes, the full master hash over their
sorted mapping, and report_sha256 over canonical JSON without that field.
Choose --format json (default), --format markdown, or --format bundle.
The deterministic ZIP bundle contains wave-report.json and wave-report.md.
Exactly one complete artifact is staged, then published without replacement.
Cleanup never unlinks the exposed destination, even if another writer replaces
it. Only private staging files are removed. Ordinary local hard-link support
is required; an unsupported filesystem fails before publication.
The old --markdown SECOND_PATH option now fails closed before any output.
Independent paths cannot provide a portable all-or-none transaction: use a
single ZIP bundle when JSON and Markdown must publish together. This removes
the exposed-path rollback race rather than relying on a check-then-delete.
report_status=VALID means chain integrity only. The separate coverage field
lists missing harnesses, and wave1_acceptance=NOT_EVALUATED deliberately does
not close external evidence gates. Even all four intact chains can contain
synthetic measurements. Input provenance labels appear in the table; unknown
origins remain UNVERIFIED_SOURCE_RECEIPT. Calibration scores are caller-input
measurements, not proof of a particular model execution. These unsigned chains
need an independently retained terminal anchor to detect a full rehash or a
valid-prefix truncation. No model authenticity or independent witness is implied.
Duplicate chains and mismatched hashed harness declarations are rejected.
Missing hashed identity metadata yields IDENTITY_UNVERIFIED rather than full
four-harness coverage; source declarations are not authenticated identities.
import json, pathlib
from szl_wave1_report import aggregate, render_markdown
chains = {}
for harness_dir in pathlib.Path("receipts/").iterdir():
if harness_dir.is_dir():
chains[harness_dir.name] = [
json.loads(p.read_text()) for p in sorted(harness_dir.glob("*.json"))
]
report = aggregate(chains)
print(render_markdown(report))
pathlib.Path("wave1-report.md").write_text(render_markdown(report))
Verified sample output (test fixture, not real-estate evidence)
Produced by the test suite in this repo on 2026-09-04 (fixture chains):
# SZL Wave 1 report
Master receipt hash: `bcd77e3ed20be9c3...`
## Chains
- `szl-engine-bench` — chain valid (2 receipts)
- `szl-retrieval-bench` — chain valid (2 receipts)
## Measured
| Harness | Lane | Metrics |
|---|---|---|
| szl-engine-bench | laneA | tok_per_s=51.0, ttft_p50_ms=1.5 |
| szl-retrieval-bench | bm25 | tok_per_s=51.0, ttft_p50_ms=1.5 |
## Blocked
- `szl-engine-bench/vllm` — endpoint not set
- `szl-retrieval-bench/vllm` — endpoint not set
The fixture numbers above are pipeline checks. The real Wave 1 table appears only when the four harnesses have emitted receipts from real runs on real hardware — tracked by each harness's verification-gate issue.
Scope
- Python 3.11+, standard library only.
- No network access, no mutation of source receipts.
- This tool verifies and formats; it never re-measures or closes provider/evidence gates.
- Explicit invalid, failed, blocked, or unavailable states cannot be promoted by stale metrics.
License
Apache-2.0 — see LICENSE.
Estate on Hugging Face: models, datasets and Spaces are published under the SZLHOLDINGS org.
Metadata
Release files for szl-wave1-report 0.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| szl_wave1_report-0.2.1.tar.gz | 21.9 kB | With provenance | Details for szl_wave1_report-0.2.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| szl_wave1_report-0.2.1-py3-none-any.whl | Python 3 | none | any | Details for szl_wave1_report-0.2.1-py3-none-any.whl |
Total release size: 37.5 kB
Release files / szl_wave1_report-0.2.1.tar.gz
| Download URL | szl_wave1_report-0.2.1.tar.gz |
|---|---|
| Size | 21.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e9852e619aa6e6c4cebf274fcdbded4f5f2d5abbc6bb360a7c77acdbde4f4d1c
|
|
BLAKE2b-256 checksum How to use checksums |
79bfd05c2b31ba20763236880d9cfa4c682c8aff33017ab609e74f008e77cbc1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.
Transparency logRelease files / szl_wave1_report-0.2.1-py3-none-any.whl
| Download URL | szl_wave1_report-0.2.1-py3-none-any.whl |
|---|---|
| Size | 15.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b2069067819af513baad6b271bed9c5f886bb9eac0e421c66f70c76aee1ac3b8
|
|
BLAKE2b-256 checksum How to use checksums |
c965bcd51a9874a3f5881af9213cef024736b6ef9dca3d1f48f3c1e6dc69a7ac
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.
Transparency logSecurity for szl-wave1-report 0.2.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for szl_wave1_report-0.2.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.