probity-verify

An offline verifier that decides whether a claim holds against evidence bytes the consumer pinned, with adapters for source passages, operand lineage, authority captures, bounded absence and spend journals.

It's for auditors, incident reviewers and relying parties who receive a claim about an AI system and need a decision they can replay, not a report they have to trust.

Install the released package: pip install probity-verify==0.2.2.

Quick start

Check a sample claim with the released package. You need Python 3.11 or newer and uv; there is nothing to clone, and a cold run takes about seven seconds. Version 0.2.2 ships the sample case and policies inside the wheel:

samples="$(uvx --from probity-verify==0.2.2 python -c 'import importlib.resources as r; print(r.files("probity_verify") / "samples")')"
uvx probity-verify==0.2.2 "$samples/case.json" --policy "$samples/policy-june.json"

It prints a decision packet that opens:

Probity decision packet v1
Case: "source-coverage-demo"
Claim: source_text_coverage/v1
Decision: contradicted
Reason: selected_span_missing_from_record
Policy SHA-256: dfda1551714f92702169d1411af6b0bfeee54732f934ce71265534af775fa388

A selected passage is in the source capture but missing from the stored report. The packet goes on to list every artifact binding and check. Swap in policy-april.json for not_established, or case-complete.json with policy-complete.json for supported. To run from a pinned source checkout instead, see Install a pinned version.

Use the decision in a check

All valid decisions exit 0, including contradicted and not_established. Exit 2 reports invalid input without a verdict. Require support for this claim by reading decision:

uvx probity-verify==0.2.2 "$samples/case.json" \
  --policy "$samples/policy-june.json" --json > decision.json &&
python3 - <<'PY'
import json

with open("decision.json") as stream:
    result = json.load(stream)
print(result["decision"], result["reason"])
raise SystemExit(0 if result["decision"] == "supported" else 1)
PY

This guard prints contradicted selected_span_missing_from_record and exits 1. supported means the stated claim passes the pinned policy's checks. Your application still owns authorization.

Status

Version 0.2.2 is on PyPI and ships the sample inputs used above. See release instructions for checked artifacts and the changelog for what each release changed. Thirteen adapters ship in 0.2.2: brokered_file_write/v1, source_text_coverage/v1, operand_lineage/v1, authority_anchor/v1, event_absence/v1, event_absence/v2, spend_reservation/v1, the done-receipt checks for coding agents protected_paths/v1, acceptance_observed/v1, tool_claim_match/v1, completion_claim/v1 and probe_observed/v1, and done_receipt/v2, which derives a done receipt's two verdicts again from them: whether the work is done and whether the agent gamed the process. Each decision is supported, contradicted or not_established. The verifier checks supplied bytes only: it does not fetch URLs or authenticate the authority a policy names.

Documentation

License

Apache-2.0.