evidence-gate

evidence-gate catches green CI verdicts whose audit trail is missing, late, or unclear.

It is a small Python library for GitHub Actions evidence bundles. It reads files that your pipeline already captured, then reports whether the evidence trail is complete, temporally bounded, and explicit about scope. It does not fetch from GitHub, run jobs, validate claim semantics, or decide whether the underlying CI result is correct.

The v0.1.0 bundle contract expects root metadata files (current_claim.json, RUN_META.json, RUN_STATUS.json, timestamp_provenance.json, and lineage_manifest.json), raw evidence under raw_evidence/, projected evidence under projected_evidence/, and verdict artifacts under verdict/. Hash manifests are supported through ArtifactExpectation, but they are optional in the default v0.1.0 completeness contract.

The Four Patterns

Completeness Check

A CI verdict can be fact-correct while its audit trail is broken. The completeness check runs after your existing verdict and verifies that the bundle contains the expected raw evidence, projected evidence, lineage, timestamp, verdict, and quarantine artifacts. It reports exact unsatisfied condition names so a caller can fail closed without guessing what is missing.

Capture-Window Timestamp Ordering

Evidence captured after a claim should not silently support that claim. The timestamp check recomputes whether raw authority was fetched inside the declared capture window and by the claim time. The completeness check uses this recomputation to catch timestamp provenance files whose stored booleans do not match their recorded fetch times. The report names that failure as timestamp_provenance_self_consistent.

A record_fetch(label, timestamps) helper is available for building the raw_github_fetch_times mapping that timestamp_provenance.json consumes. It writes a UTC timestamp into a caller-owned dict; the caller decides when to serialize the dict to disk.

Raw vs Projected Evidence

Raw API responses and derived gate facts serve different purposes. This package keeps them in separate trees and writes per-file projection lineage that names which source files produced each projected artifact. That makes the derived facts reviewable without mutating the raw capture.

Scope Bundle

An evidence bundle should state what it owns and what it does not claim. The scope helpers validate and write owned_scope, boundary_limits, and honesty_credits files. Empty honesty credits are allowed but reported as a warning.

Minimal Usage

From a clone:

python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install -e .

After publication:

python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install evidence-gate

Save this script in the clone root to try the included fixture:

from pathlib import Path

from evidence_gate.completeness import check_completeness

run_dir = Path(__file__).resolve().parent / "tests" / "fixtures" / "sample_pass_run"
report = check_completeness(run_dir)

print(report.completeness_verdict)
print(report.unsatisfied_conditions)

Integrating With An Existing Gate

from pathlib import Path

from evidence_gate.completeness import check_completeness

existing_ci_verdict = "PASS"
evidence_report = check_completeness(Path("/path/to/evidence-run"))
acceptable_evidence_verdicts = {"complete", "complete_with_quarantine"}

if (
    existing_ci_verdict == "PASS"
    and evidence_report.completeness_verdict not in acceptable_evidence_verdicts
):
    raise SystemExit("CI passed, but the audit trail is incomplete")

Bundle Sequence

For the default v0.1.0 contract:

1. Capture raw GitHub Actions files under raw_evidence/.
2. Write root metadata files: current_claim.json, RUN_META.json, RUN_STATUS.json, and timestamp_provenance.json.
3. Run project_github_run(raw_evidence_dir, projected_evidence_dir, ...). The raw directory must be named raw_evidence for the helper to write root lineage_manifest.json.
4. Call write_scope_bundle(...) to write user-authored scope files.
5. Write verdict artifacts under verdict/.
6. Run check_completeness(run_dir).

evidence-gate is tied to the v0.1.0 root metadata contract above. Within that contract, pass ArtifactExpectation to check_completeness to adapt required raw, projected, verdict, optional hash, or lineage-covered files for another bundle shape.

Claim semantics are caller responsibility in v0.1.0. project_github_run() preserves the caller's declared_claims in projected artifacts, but it does not infer whether those claims are semantically supported by the raw GitHub JSON.

Quarantine Rule

An empty quarantine/ directory is accepted. A non-empty quarantine/ directory must include one of:

• README.md
• EXPLANATION.md
• quarantine_explanation.json

Without one of those files, the completeness report marks quarantine_empty_or_explained as unsatisfied.

What This Is Not

• Not a CI runner.
• Not a replacement for branch protection.
• Not a verifier for whether the CI verdict itself is correct.
• Not a network client.
• Not a multi-platform adapter layer in v0.1.0.

Status

evidence-gate is in pre-release alpha extraction. Maintenance commitment for v0.1.0: fix correctness bugs, keep the public API small, and avoid adding platform support beyond GitHub Actions until there is clear demand.

Acknowledgments

These patterns came from repeated review of GitHub Actions evidence bundles and the failure modes that made those bundles hard to audit.