repro-evidence-kit 0.4.1

Reproducible artifact manifests, sandbox-output verification, and evidence-bundle validation.

repro-evidence-kit

A small command-line toolkit for reproducible artifact verification in binary analysis, security research, and automation workflows.

It creates hash manifests, compares experiment outputs, and validates evidence bundles so results can be reviewed without relying on private source data.

Why this matters

Generated artifacts are hard to review when the only proof is a large log, a private input tree, or a verbal claim that "nothing important changed." repro-evidence-kit keeps the review surface small: it records byte hashes, separates expected output changes from unexpected ones, and stores enough command context for another maintainer to rerun or challenge the evidence.

The project is intentionally target-neutral. It should help maintainers in CI, security research, binary-analysis, data-processing, and automation workflows without requiring them to publish proprietary samples or project-specific case files.

Use cases

• Review what changed during artifact-heavy CI or release automation.
• Verify that a sandboxed experiment only produced explicitly allowed outputs.
• Attach compact, hash-backed evidence bundles to pull requests or research notes.
• Keep generated reports reviewable without publishing private input data.

What this proves

• File manifests prove byte identity for the files they list.
• Manifest diffs separate expected artifact changes from unexpected ones.
• Sandbox verification proves the observed output set stayed inside an explicit allowlist.
• Evidence bundles preserve command context, inputs, outputs, and hashes for review.
• Signed sidecars add local tamper detection for exact bundle bytes.

What this does not prove

• Hashes do not prove that generated outputs are semantically correct.
• A passing sandbox check does not prove that a command was safe.
• Signed sidecars do not prove signer identity, key trust, command execution, or artifact semantics.
• Private or proprietary inputs still require reviewer judgment outside this repository.

Features

• Create deterministic SHA-256 manifests for files or filtered directory trees.
• Diff two manifests to identify added, removed, changed, and unchanged artifacts.
• Verify sandbox/experiment outputs against explicit allowlists, with optional JUnit XML for CI report consumers.
• Validate simple YAML or JSON evidence bundles, with optional JSON Schema checks.
• Sign and verify evidence bundle sidecars with a local-key tamper-detection prototype.
• Includes only synthetic public examples.

Install

Until a package index release is published, install from the repository:

pip install "git+https://github.com/xodnr927-byte/repro-evidence-kit.git@v0.4.1"

For local development:

python -m venv .venv
. .venv/bin/activate
pip install -e .

Quick start

repro-evidence manifest create examples/dummy-binary -o before.json
repro-evidence manifest diff before.json before.json
repro-evidence evidence validate examples/evidence-bundle.yaml

For larger artifact trees, filter manifests with explicit include/exclude patterns:

repro-evidence manifest create artifacts --include reports --exclude "*.tmp" -o manifest.json

For stricter evidence-bundle checks, install the optional schema extra and validate against the checked-in JSON Schema:

pip install "repro-evidence-kit[schema] @ git+https://github.com/xodnr927-byte/repro-evidence-kit.git@v0.4.1"
repro-evidence evidence validate examples/evidence-bundle.yaml --schema

Signed bundle sidecars are optional. For a local tamper-detection prototype, create or provide local trust material and keep it out of git:

printf 'synthetic local test key only\n' > local-test.key
repro-evidence evidence sign examples/evidence-bundle.yaml --key local-test.key -o evidence-bundle.yaml.sig.json
repro-evidence evidence verify-signature examples/evidence-bundle.yaml --signature evidence-bundle.yaml.sig.json --key local-test.key

Sandbox verification compares a baseline manifest with an after-run manifest:

repro-evidence verify sandbox-run before.json after.json --allow-added report.json

The command exits 0 when all changes are allowed and 1 when unexpected changes are present.

Documentation

• CLI reference
• CLI exit codes
• Tutorial
• Evidence bundle format
• Use cases
• Signed evidence bundles design note
• Maintainer workflow
• Release checklist
• PyPI publishing
• GitHub Actions cookbook — CI recipes for validation, manifests, sandbox checks, and schema-backed filtered workflows.
• Design principles
• Roadmap

Data policy

This repository is for generic reproducibility tooling. Do not add proprietary binaries, private datasets, copyrighted samples, live credentials, forensic case data, or project-specific reverse-engineering targets. Public examples must be synthetic or clearly redistributable.

Status

0.4.x is an early maintainer-tooling release series. The CLI and schema stay intentionally small, conservative, and synthetic-example-only.