Skip to main content

agent-evidence-replay

Replay a lab-produced agent action record: recompute its digest chain and flag a changed step (tamper) or a missing one (gap).

It is for evaluators and auditors who receive an action record from an AI lab and need to know the record is whole before they read it.

pip install "git+https://github.com/probityai/probity@agent-evidence-replay-v0.1.0#subdirectory=packages/agent-evidence-replay"
git clone --branch agent-evidence-replay-v0.1.0 https://github.com/probityai/probity && cd probity/packages/agent-evidence-replay
agent-evidence-replay verify tests/fixtures/accept-3
agent-evidence-replay verify tests/fixtures/gap-2 --pin tests/fixtures/accept-3/pin.json

The first record is a real three-step chain from the AI Agent Action conformance suite. It replays intact and the pin proves where it ends. The second is that chain with its middle step removed:

intact: 3 steps, end proven
gap: 2 steps, end proven
  link_broken [gap] line 1 r3: previousHash is '7c6c5a39...', expected 'd85166b0...'; the pin names step 'r2' here
  step_missing [gap] r2: a pinned step is absent

Exit 0 means intact, 1 means tampered, gap or broken, 2 means the record cannot be read. --json prints the findings as JSON.

How to use it

  1. When you first receive a record, make a pin and keep it where the lab cannot reach: agent-evidence-replay pin record.jsonl > pin.json.
  2. Replay every later copy against that pin: agent-evidence-replay verify record.jsonl --pin pin.json --require-end.
  3. A pin made from the bundle you are checking proves nothing. Whoever changed the record changed the pin.

A chain with no pin can still say where it breaks, but not whether a step was removed or altered. With a pin it names the cause.

Status

Version 0.1.0. The digest chain, the pin and the tamper and gap findings work and are tested against a real record and edits of it. The chain is unsigned, so a whole new consistent chain replays intact; check the lab's signature separately. Re-running the recorded tool calls is not built yet.

Documentation

page read it for
Specification the record format, every check and finding code, the pin, and what a replay does not establish
Fixtures where the test records come from

Next

  • ZIP bundles that also pin the artifacts a step names (SPEC section 8, layer 1): agent-evidence-replay verify bundle.zip.
  • Signed pins and witness heads (layer 2).
  • Re-execution of recorded tool calls against a recorded environment (layer 3).

Metadata

Release files for agent-evidence-replay 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for agent-evidence-replay 0.1.0
File Size Uploaded Provenance Actions
agent_evidence_replay-0.1.0.tar.gz 14.0 kB Without provenance Details for agent_evidence_replay-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for agent-evidence-replay 0.1.0
File Interpreter ABI Platform Actions
agent_evidence_replay-0.1.0-py3-none-any.whl Python 3 none any Details for agent_evidence_replay-0.1.0-py3-none-any.whl

Total release size: 26.1 kB

Release files / agent_evidence_replay-0.1.0.tar.gz

Download URL agent_evidence_replay-0.1.0.tar.gz
Size 14.0 kB
Tags Source
SHA-256 checksum
How to use checksums
1c7eea97583f1ff62f5a85512456f41436c9fe2158b4a7e8722187036058e285
BLAKE2b-256 checksum
How to use checksums
c995aacbf45eb191907adcd2f1f39f1a4621f1fe1a3281e025de613ff7086b33
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / agent_evidence_replay-0.1.0-py3-none-any.whl

Download URL agent_evidence_replay-0.1.0-py3-none-any.whl
Size 12.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
51efffd36ccef56ea18c86eaaf26b76beaf8265881b61c3f48f18f3b7b2f5ddb
BLAKE2b-256 checksum
How to use checksums
e62ac8d9b9ea5b1e6a99c70184178bad09e411bb051dfeaed8e6e96844ad0c5b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for agent-evidence-replay 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page