assurance-cli
Did the job cover everything it was supposed to cover?
One command. One honest ratio. An exit code your pipeline can act on.
pip install assurance-cli
assurance diff --expected corpus.txt --found retrieved.json \
--scope "documents the question spans" --where "the retrieved set" --fail-on-gap
2 of 5 documents the question spans — not in the retrieved set: doc-2, doc-3, doc-5
also present and not expected: doc-9
That second line matters as much as the ratio: the retriever drew on something the scope never allowed. It's reported, and it earns no credit.
No account · no API key · no network call · no model decides any of it
Three commands
assurance diff — any two sets of keys
# code review agent actually read the diff?
git diff --name-only origin/main...HEAD > changed.txt
assurance diff --expected changed.txt --found reviewed.txt --fail-on-gap
# eval suite ran every declared case?
assurance diff --expected cases.json --found ran.json --fail-on-gap
# straight from a pipe
retriever --query "$Q" | jq -r '.chunks[].doc_id' | \
assurance diff --expected corpus.txt --found - --json
Inputs are whatever you already have: one key per line, a JSON array (strings, or objects
with key/id/name/path), - for stdin, or an inline comma list.
assurance check — a folder of dated or numbered files
assurance check ~/reports
22 of 24 months from 2024-01 to 2025-12 in reports — not in this folder: March 2025, July 2025
— Range inferred from filenames: earliest 2024-01, latest 2025-12. Override with --from / --to.
assurance check ~/invoices --expect numbered
7 of 8 runs from inv_0001 to inv_0008 in invoices — not in this folder: INV-0006
— Range inferred from filenames: earliest inv_0001, latest inv_0008. Override with --from / --to.
Monthly, quarterly, weekly, daily, numbered. That last line is the derivation: it prints with every ratio so you can disagree with the denominator, not just the result.
assurance init — did anything change underneath?
assurance init ~/thesis-data
# ... weeks pass, several people touch the folder ...
assurance check ~/thesis-data --against-baseline
Exit codes
0 |
ran, no gap — or not asked to fail on one |
1 |
a finding: a gap with --fail-on-gap, or a baseline that no longer holds |
2 |
could not run: bad path, unreadable list, unparseable JSON |
Diagnostics go to stderr, results to stdout, so --json stays pipeable.
Use it for
| expected | found | |
|---|---|---|
| RAG | documents the question spans | chunks retrieved |
| Code review in CI | git diff --name-only |
files reviewed |
| ETL / batch | records or partitions declared | records or partitions loaded |
| Compliance | controls in scope | controls with evidence |
| Research data | the series you expect | what's actually in the folder |
What it won't do
- Invent your expected set.
difftakes your declaration;checkderives one and prints how - Send anything anywhere. No network, no telemetry, no keys
- Guess. A JSON object of id → metadata is refused, not interpreted
Family
assurance-core — the pure arithmetic, zero dependencies · assurance-mcp — the same checks as MCP tools
Upstream is I-Ops; this repo is a publication, never a source. Apache-2.0.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file assurance_cli-0.2.1.tar.gz.
File metadata
- Download URL: assurance_cli-0.2.1.tar.gz
- Upload date:
- Size: 22.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
055dd11e79107e2cc775f83fc6820d46003b09dd976bf230c06708c665cb8140
|
|
| MD5 |
b5bcbb05d5e13dcb916140e6598490b0
|
|
| BLAKE2b-256 |
1a8292661d82e9ff0676e7983b8a01b6c50ebbbd12e84359f8c2099a5ac7ff2b
|
Provenance
The following attestation bundles were made for assurance_cli-0.2.1.tar.gz:
Publisher:
publish.yml on i-ops-hq/assurance-cli
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
assurance_cli-0.2.1.tar.gz -
Subject digest:
055dd11e79107e2cc775f83fc6820d46003b09dd976bf230c06708c665cb8140 - Sigstore transparency entry: 2643577431
- Sigstore integration time:
-
Permalink:
i-ops-hq/assurance-cli@87f39cf79675e574e8eb24aba2d8a81a3f126d85 -
Branch / Tag:
refs/tags/v0.2.1 - Owner: https://github.com/i-ops-hq
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@87f39cf79675e574e8eb24aba2d8a81a3f126d85 -
Trigger Event:
push
-
Statement type:
File details
Details for the file assurance_cli-0.2.1-py3-none-any.whl.
File metadata
- Download URL: assurance_cli-0.2.1-py3-none-any.whl
- Upload date:
- Size: 19.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7cf26f4446046d4edeb55cdaebbc8c0c77f84b951f1aa6d301db540d2a730229
|
|
| MD5 |
e7683840b3f03d9dd0aa4f42fd263398
|
|
| BLAKE2b-256 |
3f75cc67532dd2a8f82abccd15528fb5094e597603619ee0d32dbbedf2c45bfc
|
Provenance
The following attestation bundles were made for assurance_cli-0.2.1-py3-none-any.whl:
Publisher:
publish.yml on i-ops-hq/assurance-cli
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
assurance_cli-0.2.1-py3-none-any.whl -
Subject digest:
7cf26f4446046d4edeb55cdaebbc8c0c77f84b951f1aa6d301db540d2a730229 - Sigstore transparency entry: 2643577853
- Sigstore integration time:
-
Permalink:
i-ops-hq/assurance-cli@87f39cf79675e574e8eb24aba2d8a81a3f126d85 -
Branch / Tag:
refs/tags/v0.2.1 - Owner: https://github.com/i-ops-hq
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yml@87f39cf79675e574e8eb24aba2d8a81a3f126d85 -
Trigger Event:
push
-
Statement type:
