hanzo-research
File a measurement made in Python to Hanzo's research plane (/v1/research) and its
benchmark arena (/v1/benchmark), with what it takes to believe or repeat it: the
harness commit, the model and its weights' SHA-256, the host, CPU and GPU, the device and
dtype, the Python and package versions, the date. The Python peer of the Rust crate
hanzo-research (hanzoai/ml): the same wire records, the same RFC 8785 digests, the same
lineage and WAL records. Standard library only.
pip install hanzo-research # or: uv run --with hanzo-research python measure.py
The credential is HANZO_API_KEY, else what hanzo auth token prints (a CLI token is
fetched again when it expires). Records land in the token's org (its owner) and project
(its project claim, else default). Research(org="lux") acts in another org the
token belongs to (X-Org-Id); an org it does not belong to is refused before any request,
since the server would file under the home org instead.
Worked example
Measure something, file it with its provenance, make it public, read it back.
import hashlib
import platform
import time
from hanzo_research import Benchmark, Research, Run, Split, capture, now
r = Research()
bench = Benchmark(
id="demo.sha256",
title="SHA-256 over 16 MiB",
splits=[Split("16mib", 30, "30 hashes of 16 MiB of zeros, one process, after one discarded")],
metric="p50_ms",
metrics=["p50_ms", "p95_ms", "mb_s"],
)
p = capture(device="cpu") # harness commit, host, CPU, GPU, Python, packages, date
start = now()
data = bytes(16 << 20)
hashlib.sha256(data).digest()
ms = []
for _ in range(30):
t = time.perf_counter()
hashlib.sha256(data).digest()
ms.append(1000 * (time.perf_counter() - t))
ms.sort()
run = Run(
id=f"demo.sha256-16mib-cpu-{p.host}",
benchmark=bench.id,
split="16mib",
system="hashlib",
version=platform.python_version(),
questions=30,
answered=30,
when=start,
ended=now(),
)
run.put("all", "p50_ms", ms[14], n=30, of=30)
run.put("all", "p95_ms", ms[28], n=30, of=30)
run.put("all", "mb_s", 16 / (ms[14] / 1000))
p.apply(run) # fills commit and by, appends the provenance sentence and its digest to notes
r.record(benchmarks=[bench], runs=[run]) # the definition, then the run: both private
r.grant(run.id, visibility="public", publishable=True)
back = r.get_run(run.id, org=r.org) # the public read: no credential
print(back.completion, back.measure("all", "p50_ms").value, back.commit)
completion is the server's: complete only when answered == questions and ended is
set; partial, inconsistent or unknown otherwise. A model run names its weights:
capture(model="kai-1.3", weights="/models/kai-1.3", device="cuda", dtype="bf16") hashes a
file, or a directory by its SHA256SUMS (else every file under it), and apply sets
system to the model and version to <model>@<sha256>. A dirty harness tree is recorded
with the SHA-256 of git diff HEAD and the untracked files, so the number stays pinned to
the code that made it.
The research plane: Research
| call | route |
|---|---|
record(benchmarks=, studies=, runs=, papers=) |
POST /v1/research/{benchmarks,studies,runs,papers}, one batch a non-empty kind, in that order: {"runs": [...]} |
benchmarks(id=, project=), get_benchmark(id) |
GET /v1/research/benchmarks |
runs(id=, benchmark=, system=, study=, completion=, since=, until=, project=) |
GET /v1/research/runs: your org's runs, private ones included, with the completion census and the fields runs leave unstated |
runs(org=...), get_run(id, org=...) |
the same route with ?org= and no credential: that org's public runs |
studies(...), papers(..., cites=) |
GET /v1/research/{studies,papers} |
compare(a, b) |
GET /v1/research/compare: the reasons the two runs may not be compared, the config they differ in, the deltas |
grant(id, visibility=, publishable=, trainable=), grant(sha256=, visibility=) |
POST /v1/research/grants: on every version of the id; a write is always private until this |
revise(record) |
a new version marked corrected |
retract(id, why) |
a new version marked retracted: the id leaves every listing, earlier versions stay |
ingest, query, totals, projects, experiment(...) |
the experiment plane: /v1/research/{experiments,totals,projects} |
upload_artifact, upload_json_artifact, artifacts, get_artifact |
/v1/research/artifacts[/<sha256>]: snapshot or report bytes, addressed by the server's hash |
url(run_id, org=) |
the public read address of one run: the source a claim cites |
Every write is a batch of at most 20,000 and is idempotent by content: the same record
again records nothing. A mapping is sent as given; a Run is sent with every field. A
refusal raises Denied (401, 403), Missing (404), Invalid (400, 409, 422), Limited
(413, 429) or Unavailable (502 to 504, no answer), each carrying the server's own
sentence; a 503 is asked again after 2, 4 and 8 seconds. A NaN or infinity never leaves:
json refuses it before the request.
The arena: Arena (or Research().arena)
| call | route |
|---|---|
catalog() |
GET /v1/benchmark/catalog |
leaderboard(benchmark) |
GET /v1/benchmark/leaderboard: each model's measured score (its latest run) beside a published claim. Only claims a SuperAdmin wrote under org admin, and public, reach it. |
claims(benchmark=, model=, provider=, source=, protocol=, org=) |
GET /v1/benchmark/claims: every org's public claims, labelled, and your org's private ones |
post_claims(rows, visibility=) |
POST /v1/benchmark/claims: under your acting org; each row names its protocol and its source (for a measured score, Research.url(run_id)), or it is refused here |
history(benchmark, model=), compare(a, b, benchmark=) |
GET /v1/benchmark/{history,compare}: read the arena's per-item attempts plane, not the research runs |
presets(), post_preset(preset) |
GET/POST /v1/benchmark/presets: validated and echoed as enso-<name>, not stored |
post_run(benchmarks, model=, endpoint=, confirm=True) |
POST /v1/benchmark/runs: queues a paid execution, so it is refused without confirm=True |
The command
hanzo-research (or python -m hanzo_research); every answer is JSON on stdout.
hanzo-research post runs.jsonl --grant public --publishable # a run per line
hanzo-research post plan.json # {"benchmarks": [...], "runs": [...]}
hanzo-research post plan.json --dry # print the bodies, file nothing
hanzo-research list runs --benchmark demo.sha256
hanzo-research list runs --org hanzo # hanzo's public runs, no credential
hanzo-research get demo.sha256-16mib-cpu-dgx --org hanzo
hanzo-research grant RUN_ID ... --visibility public --publishable
hanzo-research retract RUN_ID --why "read on the wrong split"
hanzo-research compare RUN_A RUN_B
hanzo-research provenance --model kai-1.3 --weights /models/kai-1.3 --device cpu
hanzo-research arena leaderboard --benchmark gpqa_diamond
It is not hanzo research: the hanzo command is the native CLI, and no Python package
in this workspace may install one.
Records beside the API
canonical.canonicalize(value),canonical.digest(value): RFC 8785 bytes and their SHA-256, equal to the crate'srfc8785_canonicalize/rfc8785_digeston every string, integer within 2^53 and float from 1e-6 to 1e21. Past that the crate writes Rust'sDisplay(1e21as 22 digits,-0.0as-0), which is not RFC 8785; this package writes the RFC's1e+21and0.tests/vectors/rust.jsonholds the crate's own output.lineage:DataPurpose,LineagePolicy(a child never less restrictive than a parent),ContaminationError,verify_promotion_eligibility, and the crate's evidence records (EnvironmentV1,GateSpec,Claim,TeacherInteraction,PairEvidence,DeterminismReport,ArtifactRef,ResearchEvent), each with the crate's digest.wal.ResearchWal: an fsynced, hash-chained event log. A log the crate wrote verifies here (wal.verify), and the reverse.
Tests
uv run --no-project --with pytest pytest pkg/hanzo-research # no network
uv run --no-project --with pytest pytest pkg/hanzo-research -m integration # api.hanzo.ai, no credential
Metadata
Release files for hanzo-research 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| hanzo_research-0.2.0.tar.gz | 55.5 kB | Without provenance | Details for hanzo_research-0.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| hanzo_research-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for hanzo_research-0.2.0-py3-none-any.whl |
Total release size: 95.4 kB
Release files / hanzo_research-0.2.0.tar.gz
| Download URL | hanzo_research-0.2.0.tar.gz |
|---|---|
| Size | 55.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9935c71caaa4df46f308e03f79b09442b6bcb964180b8d551cb553e3338e721d
|
|
BLAKE2b-256 checksum How to use checksums |
44597cc8665d140c85e162713083ff0a6ee9274161acc1bc7421f4a70233f8f8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.9
|
Release files / hanzo_research-0.2.0-py3-none-any.whl
| Download URL | hanzo_research-0.2.0-py3-none-any.whl |
|---|---|
| Size | 39.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b63accd37065015ecaefe61ba5dbf13e3de387aa1de3c0f0c5559165922e0c79
|
|
BLAKE2b-256 checksum How to use checksums |
b9bb3746bf0614d74b29e6d17e3fd670a91a46fdb9b6c99a4641c224adb482ac
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.9
|
Security for hanzo-research 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.