Probity Vectors

Test an agent-evidence verifier against saved records with known outcomes. The corpora cover valid records, altered bytes, missing evidence and open specification interpretations.

Try it

From a reviewed source checkout, build the Go verifier and a wheel, then run the installed harness outside the checkout. You need Go 1.24+, Python 3.13+ and uv. Record the checkout's full commit ID with your report.

git rev-parse HEAD
run_dir="$(mktemp -d)"
GOWORK=off go build -o "$run_dir/aee-verify" ./cmd/aee-verify
uv build --wheel --out-dir "$run_dir"
wheel="$(find "$run_dir" -maxdepth 1 -name 'agent_evidence_vectors-*.whl')"
cd "$run_dir"
uv run --no-project --with "$wheel" agent-evidence-vectors --verifier "$run_dir/aee-verify --json"

Open conformance-report.json for the per-vector results. A complete external run has rail: external, verifier.vectorsExecuted equal to totals.vectors, and zero totals.suiteRefusals. The report separates conformance from reason-code agreement.

See installation routes and release identity for package availability and signature checks.

After the signed tag and PyPI wheel are published, install the matching release (with Go's executable directory on your PATH):

go install github.com/probityai/agent-evidence-vectors/cmd/aee-verify@v0.17.4
uvx agent-evidence-vectors==0.17.4 --verifier "aee-verify --json"

Test your verifier

The harness calls your executable with a vector-file path. Return a verdict through the exit status and a JSON object on the last line of stdout. The verifier contract covers output fields, key policy and comparison rules.

Use the GitHub Action to run the same check in your build and retain its report. The CI guide gives a workflow step and the available inputs.

Choose your next step

Each corpus manifest records its source revision, digest and expected outcomes. DISTRIBUTION.md lists the shipped corpora and installation routes.

For fixture coverage, see Corpus measurements. For outside results, see External records. To check signed release bytes, use Release verification.

Source maps: repository layout, corpus readers and witness attestor.

Contribute

Open an issue with the corpus revision, verifier build and report. For patches, start with CONTRIBUTING.md. For citations, use the citation guide.

Need to check a supplied claim against saved evidence? See Probity Verify. Browse the Probity projects for records, vocabulary and admission tools.

Apache-2.0. LICENSE.