inspect-receipts 0.2.0

Emit ZAP1 verifiable attestation receipts from Inspect AI evals (verify-without-trust eval verdicts).

inspect-receipts

Emit ZAP1 verifiable attestation receipts from Inspect AI evals. Install the package, run any eval, and a signed-evidence receipt for the run lands next to your logs with no fork and no code change to your tasks.

What this is: the missing signed-evidence layer for agent evals. Inspect (and the broader agentskills.io ecosystem) produce rich results but ship them as trust-on-read JSON. inspect-receipts turns each eval into a hash-commitment: input_hash (what was evaluated), output_hash (the verdict), and locked_harness_hash (the frozen judge), so a third party can recompute without trusting the runner.

What this is NOT: a proof that the eval, the scorer, or the system-under-test is correct. It is a tamper-evident commitment, not a correctness SNARK. No prompts, completions, transcripts, or secrets ever enter a receipt: only canonical SHA-256 hashes, numeric metrics, and coarse identifiers.

Install

pip install inspect-receipts          # plus: pip install inspect_ai

Inspect auto-discovers the hook via its inspect_ai entry-point group. There is nothing to import in your eval.

Use

inspect eval my_task.py --model anthropic/claude-opus
# -> ./zap1-receipts/receipt-<task_id>.json

Each completed task writes a receipt like:

{
  "schema": "zap1.vwc.v0/attestation",
  "receipt_family": "AGENT_EVAL_VERDICT",
  "vwc_profile": "attestation",
  "subject": { "kind": "inspect_eval", "task": "ipi_resistance", "model": "..." },
  "hashes": {
    "input_hash":  "sha256:...",
    "output_hash": "sha256:...",
    "locked_harness_hash": "sha256:..."
  },
  "score": { "kind": "scalar", "direction": "maximize", "name": "match.accuracy", "value": 0.92 },
  "metrics": ["match.accuracy=0.92"],
  "status": "success",
  "public_boundary": "hash-commitment, NOT a proof of correctness"
}

Verify (without trusting the runner)

The whole point is that a third party can recompute the commitments straight from the published eval log — with nothing installed but this package (no inspect_ai needed to verify a JSON-dumped log):

inspect-receipts-verify ./logs/my_run.eval ./zap1-receipts/receipt-<task_id>.json
#   [OK ] input_hash
#   [OK ] output_hash
#   [OK ] locked_harness_hash
#
#   MATCH: receipt commits to this eval log (hash-commitment verified).

Exit code is 0 on a full match, 1 on any mismatch — so it drops straight into CI. If someone edits the verdict after the fact, the output_hash line flips to XX and prints both the claimed and recomputed digest, localizing the tamper. The receipt carries the (non-secret) judge identity in cleartext, so the locked_harness_hash pre-image is reconstructable without the runner's help.

This verifies the commitment — that the receipt is bound to this log — not that the eval's verdict is correct. From Python:

from inspect_receipts import verify_receipt
verify_receipt(eval_log, receipt)["ok"]   # True / False

Configuration (env)

Variable	Default	Effect
INSPECT_RECEIPTS	1	Set 0 to disable receipt emission.
ZAP1_RECEIPT_DIR	./zap1-receipts	Where receipts are written.
ZAP1_RECEIPT_ISSUER	inspect-receipts	Issuer label in the receipt.
ZAP1_RECEIPTS_ANCHOR	0 (off)	Opt-in: POST the receipt to a ZAP1 endpoint.
ZAP1_ENDPOINT / ZAP1_API_KEY	unset	Required only if anchoring is enabled.

Default is local-only / no network. Anchoring is opt-in and transmits only the bounded receipt (hashes + metrics), never eval content. Verify locally before you anchor.

Receipt format

Receipts use the ZAP1 attestation schema (zap1.vwc.v0/attestation): a locked, hashed judge (the Inspect task, model, and config), a recomputable verdict, and three bound hashes. The same envelope is reused for benchmark and gate receipts, so one verification path covers all of them.

License

MIT.