Certified Corruption Budget
Attach a certified corruption budget to leaderboard claims.
Every ranking claim gets a number B_hat: with probability at least 1 - alpha,
at every time t, either the claim holds, or more than B_hat of the first
t records were corrupted. The guarantee holds even if an attacker watches
every published certificate and corrupts votes in bursts.
This is a product implementation of:
Hamed Khosravi and Xiaoming Huo, Certified Corruption Budgets: Anytime-Valid Leaderboard Claims under Adaptive Rigging, arXiv:2610.10597, 2026. https://arxiv.org/abs/2610.10597
This is an independent implementation. It is not affiliated with, endorsed by, or reviewed by the paper's authors.
Status
Local-first Python library and CLI. Edge and lead certificates, insertion and
replacement costs, batch and streaming, selection correction, closed-testing
and Bonferroni leaderboards, attack replays, a human table, a stable ccb/v1
JSON envelope, and a self-contained HTML report are implemented and tested.
Registry publication is pending the owner's decisions.
Install
uv build
pip install dist/certified_corruption_budget-0.1.0-py3-none-any.whl
Python 3.11 or newer; NumPy is the only runtime dependency.
Example
$ ccb certify --input records.jsonl --pair model-x,model-y
pair model-x > model-y
claim edge
cost replacement
alpha 0.05
B_rep 2
certified_at 13
At alpha=0.05, the edge claim that model-x has had an advantage over model-y holds unless more than 2 of the first 20 records were forged or altered (replacement cost).
What it does
- Ingests pairwise records from JSONL or CSV.
- Emits edge and lead certificates with insertion or replacement costs.
- Emits simultaneous certificates for a whole leaderboard.
- Corrects for selecting the best of
Vprivate model variants. - Replays burst, flip, and spread attacks against the certificate and the paper's baselines.
- Produces a human table, a stable JSON envelope, and a static HTML report.
CLI
ccb certify --input examples/records.jsonl --pair a,b --claim edge --cost replacement --alpha 0.05
ccb certify --input examples/records.jsonl --pair a,b --json
ccb leaderboard --input examples/records.jsonl --method closed --cost replacement --alpha 0.05
ccb replay --input examples/null.jsonl --attack flip --budget 100 --replays 1000 --json
ccb demo --input examples/records.jsonl --out demo.html
ccb version
ccb certify and ccb leaderboard accept --json and print exactly one
ccb/v1 envelope. Schemas live in schema/ and ship inside the wheel.
Input shape
One record per line:
{"model_a":"model-x","model_b":"model-y","outcome":1.0,"t":0}
outcome is 1.0 if model_a wins, 0.0 if model_b wins, 0.5 for a tie.
t and variant are optional; --format csv reads the same columns from a
CSV header.
Limitations
- A certificate is robustness to corrupted records, not proof that a model is better, and not fraud detection.
ccb replayestimates false-statement rates by resampling the input stream; the rates are meaningful only when the uncorrupted claim is false.- A leaderboard is a snapshot at the shortest input stream. There is no streaming update of a published leaderboard.
- A
-1budget means no certificate was issued at the chosenalpha; it is not evidence against the claim. - The guarantee assumes
alpha, the horizon, the corruption cost, the pair set, and the variant count were fixed before the data was seen. Choices made after looking at the data need their own correction. - Local files only. There is no hosted ingestion, account, or database.
- The paper is a preprint; the implementation is not peer reviewed.
Security and contributing
Report vulnerabilities through a GitHub security advisory, not a public issue.
Development: uv sync, then uv run pytest and uv run ruff check .. A change
to a certificate or leaderboard formula needs a proof or a citation to the
paper; name the section in the function docstring.
Licence
MIT (see LICENSE).
Metadata
Release files for certified-corruption-budget 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| certified_corruption_budget-0.1.0.tar.gz | 17.1 kB | Without provenance | Details for certified_corruption_budget-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| certified_corruption_budget-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for certified_corruption_budget-0.1.0-py3-none-any.whl |
Total release size: 39.6 kB
Release files / certified_corruption_budget-0.1.0.tar.gz
| Download URL | certified_corruption_budget-0.1.0.tar.gz |
|---|---|
| Size | 17.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
1c02fdd1266da2fb4097ee3b9deb3417d18869be0021a3f81b7d88536f42c7e4
|
|
BLAKE2b-256 checksum How to use checksums |
17832c67c8ac61804575b9de48afbb894501dc0fc7efa403223e8e88d26091ac
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / certified_corruption_budget-0.1.0-py3-none-any.whl
| Download URL | certified_corruption_budget-0.1.0-py3-none-any.whl |
|---|---|
| Size | 22.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f0607a04ff05659ffc617a692c3887cfe1c53e64785f8195783f930b64f1720f
|
|
BLAKE2b-256 checksum How to use checksums |
d7de2a70c0b1ba4817e5c14177f2736d0eeab5e901e18f7c3eab13684d7d727d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.9 {"installer":{"name":"uv","version":"0.13.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for certified-corruption-budget 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.