mizan-ledger
Declare how confident your agent is before a task, commit a salted hash of that declaration, then score honesty against the real outcome (your own tests, evals or checks). Local-first, hash-only, verifiable offline.
Status: v0.1.3, early alpha by QANTS. On PyPI (not on npm). MIT licensed (see
LICENSE). No network access, no telemetry, no dependencies in the core library.
Why
Agents often report success with the same unearned confidence whether they are right or wrong. A calibration ledger asks one question: when the agent says "80% sure", is it right about 80% of the time? Brier score and ECE answer that. The commitment makes sure the declaration was fixed before the outcome was known.
Install
pip install mizan-ledger
Or from a local checkout:
pip install -e . # Python >= 3.9, pytest only for development
30-second example
from mizan_ledger import Ledger
led = Ledger("ledger/", private_path="private/") # public dir is safe to share; private dir is not
led.declare("fix-bug-421", 0.80, agent_id="my-agent", version="1.2.0") # hash-only receipt, timestamped
# ... run the task, run the tests ...
led.resolve("fix-bug-421", outcome=True) # reveals the opening, scores it
print(led.summary("my-agent")) # n, Brier + 95% interval, ECE, log loss, accuracy (Wilson), coverage
mizan-ledger verify ledger/ # offline check of hash chains, openings and scores
What gets recorded
| File | Visibility | Contents |
|---|---|---|
receipts.jsonl |
public | agent_id, agent_version, commitment (SHA-256), declared_at, hash-chain link |
resolutions.jsonl |
public after resolution | outcome, opening (task_id, p, salt), squared error, log loss, hash-chain link |
private/openings.jsonl |
private (mode 600) | the same openings, kept until the task is resolved |
Commitment = SHA-256(canonical_json({agent_id, agent_version, p, salt, schema, task_id})) with p as a fixed 6-decimal string, a fresh 128-bit random salt per declaration, sorted keys and no whitespace. Shared test vectors (vectors/commitments.json) are checked by both the Python and the JavaScript implementation.
Guarantees and honest limits
- A declaration cannot be replaced or re-resolved. Deleting or editing a record breaks the hash chain and
verifyfails. - Coverage is reported. Declared, resolved and pending are all counted, so you cannot quietly drop the bad ones. Anything unresolved shows as
pending. - Timestamps are your local clock. They prove order only if the chain head is anchored somewhere independent (publish
chain_head(), timestamp it, or write it to a transparency log). Without that, a ledger owner could rebuild the whole chain after the fact. - The library cannot tell whether the outcome you pass to
resolveis honest. Use an outcome source you do not control through the agent (test results, an external check). - Small samples give wide intervals; ECE is biased upward for small n. The summary prints the interval and
n; read them. - Scores are about calibration, not about whether the agent is good, safe or secure.
Scores (mizan_ledger.metrics)
Brier (mean squared error), log loss (p clipped to [0.01, 0.99], same as the Mizan API), ECE with 10 equal-width bins, reliability table, accuracy with a Wilson 95% interval, and a deterministic percentile-bootstrap 95% interval for Brier. The always-0.5 baseline Brier is 0.25.
Adapters
| Adapter | Status |
|---|---|
pytest plugin (mizan_ledger.pytest_plugin): @pytest.mark.confidence(0.9) declares before the test body, pass/fail resolves it. Run with pytest --mizan-ledger-dir .mizan --mizan-agent-id my-agent. |
Tested here (pytester). |
Generic trace evaluator (mizan_ledger.adapters.trace_eval): plain-dict functions to declare from a trace's confidence field and resolve from its score. |
Unit-tested with synthetic dicts only. Not tested against Arize Phoenix, Braintrust or LangSmith; their evaluator signatures were not verified. Treat as a stub. |
JavaScript shim (js/mizan-ledger.mjs): commitment, declare, verify opening. Node >= 18, node:crypto only. |
Tested against the shared vectors (node --test js/test.mjs). It does not implement the file ledger. |
No CI workflow file is included in this repository.
Local MCP server (optional)
An optional stdio MCP server lets an agent host (Cursor, Claude Desktop, VS Code or any MCP client) use the ledger without leaving the machine. It needs Python >= 3.10; the core install is unchanged.
pip install "mizan-ledger[mcp]"
{"mcpServers": {"mizan-ledger": {"command": "mizan-ledger-mcp",
"env": {"MIZAN_LEDGER_DIR": ".mizan", "MIZAN_AGENT_ID": "my-agent"}}}}
Tools: commit_confidence (declare before the task), resolve_outcome (record the real outcome), get_calibration (read-only scores), verify_ledger (read-only offline check). Local files only: no network, no keys, no shell commands. The outcome you supply is not verified; prefer a source the agent does not control. MIZAN_PRIVATE_DIR (default <MIZAN_LEDGER_DIR>/private) holds the openings; keep it out of version control.
This is not the remote qants-diwan MCP server on the QANTS site (Mizan benchmark and directory); that is a different product.
Tests
python3 -m pytest -q # Python suite
node --test js/test.mjs # JS shim vs shared vectors
Seed agents
seeds/ holds three demonstration agents operated by QANTS (ids start with qants-). They use only free public data and local ledgers, never the production Mizan API, and they are always shown separately from external agents. Their raw ledgers are in seed_runs/ and can be checked with mizan-ledger verify seed_runs/<agent>.
Security notes
Keep private/ out of version control and backups you share. The salt is what stops guessing of low-entropy declarations.
License
MIT, see LICENSE. Copyright (c) 2026 QANTS.
Metadata
Release files for mizan-ledger 0.1.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| mizan_ledger-0.1.3.tar.gz | 23.5 kB | Without provenance | Details for mizan_ledger-0.1.3.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| mizan_ledger-0.1.3-py3-none-any.whl | Python 3 | none | any | Details for mizan_ledger-0.1.3-py3-none-any.whl |
Total release size: 41.0 kB
Release files / mizan_ledger-0.1.3.tar.gz
| Download URL | mizan_ledger-0.1.3.tar.gz |
|---|---|
| Size | 23.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
de5989f12e4488587764c7021ba2bf069b3114cca9f4983df5ae196d78b64c47
|
|
BLAKE2b-256 checksum How to use checksums |
79afe01ba7657d24176818624970c88b5e13e87e87f0a924a422a7708a5ca617
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.5
|
Release files / mizan_ledger-0.1.3-py3-none-any.whl
| Download URL | mizan_ledger-0.1.3-py3-none-any.whl |
|---|---|
| Size | 17.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1796ab9d1b402432684518560aa80a5f237182381d2724c514ff69223227c0b6
|
|
BLAKE2b-256 checksum How to use checksums |
95b4279183ba7eae4c11c4d5c9c2c16779bac83f0d5ce71c502577b21a80c810
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.5
|
Security for mizan-ledger 0.1.3
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.