shadow-diff
Find the exact change that broke your AI agent.
Shadow is a CI-native regression-forensics tool for LLM agents. One command on the PR — shadow diagnose-pr — answers:
- Did agent behavior change?
- How many traces are affected?
- Which exact prompt / model / tool / config change caused it?
- With what confidence (ATE + bootstrap CI + E-value when run with
--backend live)? - What fix should
verify-fixconfirm before merge?
The PyPI distribution is shadow-diff. The Python import path is shadow. The CLI is shadow.
Install
pip install shadow-diff
Requires Python 3.11+. Pre-built wheels ship for Linux x86_64, macOS arm64, and Windows x86_64; other platforms build from source (Rust required).
Optional extras:
pip install 'shadow-diff[anthropic]' # if your agent uses Claude
pip install 'shadow-diff[openai]' # if your agent uses GPT
pip install 'shadow-diff[embeddings]' # paraphrase-robust semantic diff
pip install 'shadow-diff[all]' # everything
60-second tour
shadow demo # nine-axis diff on bundled fixtures, no API key
shadow quickstart # writable copy of a runnable scenario
Then run diff against the writable scenario:
cd shadow-quickstart
shadow diff fixtures/baseline.agentlog fixtures/candidate.agentlog
For the full diagnose-pr flow against your own agent, see docs/features/causal-pr-diagnosis.md and the runnable refund-causal-diagnosis demo.
Record your own agent
from shadow.sdk import Session
with Session(output_path="trace.agentlog"):
# Your existing Anthropic / OpenAI code, unchanged.
client.messages.create(model="claude-sonnet-4-6", messages=[...])
Shadow auto-instruments the Anthropic and OpenAI SDKs and writes content-addressed .agentlog files. Secrets are redacted by default. Or skip the code change entirely:
shadow record -o trace.agentlog -- python your_agent.py
Daily workflow — Shadow as pytest for agent behavior
shadow inspect trace.agentlog # debug a single trace
shadow scan baseline_traces/ # block secret leaks
shadow baseline create baseline_traces/ # pin the gold standard
shadow gate-pr ... # gate every PR
Full docs
The canonical README, the .agentlog spec, runnable examples, and the comparison against adjacent agent-eval and runtime-governance tools all live at https://github.com/manav8498/Shadow.
License
Apache-2.0. See LICENSE-APACHE in this distribution. The .agentlog spec is independently published under Apache-2.0.
1 maintainer
Metadata
1 maintainer
Release files for shadow-diff 3.2.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| shadow_diff-3.2.5.tar.gz | 581.0 kB | With provenance | Details |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| shadow_diff-3.2.5-cp311-abi3-win_amd64.whl | CPython 3.11 | abi3 | Windows x86-64 | Details |
| shadow_diff-3.2.5-cp311-abi3-manylinux_2_34_x86_64.whl | CPython 3.11 | abi3 | Linux glibc 2.34+ x86-64 | Details |
| shadow_diff-3.2.5-cp311-abi3-macosx_11_0_arm64.whl | CPython 3.11 | abi3 | macOS 11.0+ ARM64 | Details |
Total release size: 3.8 MB
Release files / shadow_diff-3.2.5.tar.gz
| Download URL | shadow_diff-3.2.5.tar.gz |
|---|---|
| Size | 581.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
01591fe64a594812db3f0edbaf7e8f1146f722386c10f551661dcb0923a27f6e
|
|
BLAKE2b-256 checksum How to use checksums |
c54f18de8d3e65ff4c35adf589270f033ecc78595fcea94ed17295bbd7b25689
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on May 13, 2026.
Transparency logRelease files / shadow_diff-3.2.5-cp311-abi3-win_amd64.whl
| Download URL | shadow_diff-3.2.5-cp311-abi3-win_amd64.whl |
|---|---|
| Size | 1.0 MB |
| Tags | CPython 3.11 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
5fc79286f8154e533e1df4b9afb7e470551a82471b6a35dfa8083bab68f7bc77
|
|
BLAKE2b-256 checksum How to use checksums |
a00c1379f0225755387f8aefb346d23b47a31ca9aca8a2de9c38c33229f98407
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on May 13, 2026.
Transparency logRelease files / shadow_diff-3.2.5-cp311-abi3-manylinux_2_34_x86_64.whl
| Download URL | shadow_diff-3.2.5-cp311-abi3-manylinux_2_34_x86_64.whl |
|---|---|
| Size | 1.2 MB |
| Tags | CPython 3.11 Linux glibc 2.34+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
70d6a8e9e00415b7351165c5ad9dc43674f533fbdcf6f70785ceb349dc29f82a
|
|
BLAKE2b-256 checksum How to use checksums |
db553361b3ae62ff443f91b2f5bc6ab9ac1dd28985de10c29b13c6400d95b4c2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on May 13, 2026.
Transparency logRelease files / shadow_diff-3.2.5-cp311-abi3-macosx_11_0_arm64.whl
| Download URL | shadow_diff-3.2.5-cp311-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | CPython 3.11 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
8699e18dab463fb95f6a3e01ae8423e164d88d9c5c14a899de3e51847988568e
|
|
BLAKE2b-256 checksum How to use checksums |
03c24a2c9bae52d25770888ac82009447f5140706cb90375591b120350e40078
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.13
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on May 13, 2026.
Transparency logSecurity for shadow-diff 3.2.5
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 4 files in this release have been uploaded with provenance metadata.
Attestation for shadow_diff-3.2.5.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on May 13, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
