auditlite

Chain-of-custody for AI-assisted analyses.

Traditional reproducibility asks: can the analysis be rerun? AI-assisted analysis adds a second question: can it be reconstructed? The final result can look coherent while the reasoning path is scattered across prompts, generated code, manual edits, reruns, copied outputs, and decisions nobody wrote down. Three months later — or one reviewer letter later — nobody can say which code the model wrote, why an exclusion happened, or whether Table 1 still comes from the current script.

auditlite keeps one append-only, hash-chained trail per project (audit/trail.jsonl) and records what changed, why it changed, who or what changed it, and whether the reported result still follows from the current analysis.

What it is not. auditlite is not an AI detector — it records declared provenance and flags where declarations are missing. An intact hash chain shows the trail has not been rewritten after the fact; it does not show the recorded events are true. Checks report support for reconstruction, never correctness of the science.

Install

pip install auditlite

Python ≥ 3.10, no dependencies.

What gets recorded

The checks

auditlite check runs seven integrity checks. Each answers three ways — ok, flag, or not_checkable — because a check that lacks the events it needs should say so rather than guess.

• chain — has the trail itself been edited, reordered, or truncated?
• script_drift — were scripts edited after their last recorded run?
• input_drift — does the data on disk still match what the runs consumed?
• stale_outputs — were outputs modified outside a recorded run?
• claim_freshness — do manuscript tables/figures still match the files they were recorded from?
• provenance_coverage — did any script execute without a declared origin?
• decision_rationale — were decisions recorded without a why (worst when the AI introduced them)?

Exit code 1 if anything is flagged, so it drops into CI or a pre-submission checklist as-is.

Workflow

auditlite init --project lung-registry-survival

# declare provenance when code enters the project
auditlite provenance analysis.py --origin ai_generated \
    --model claude-sonnet-5 --prompt "primary survival analysis per SAP v2" \
    --context sap_v2.md --context codebook.csv

# trace analytic decisions as they happen
auditlite decision --what exclusion --choice "drop ECOG 4" \
    --why "n=3 in stratum, prespecified minimum is 10" \
    --source ai_suggested --disposition edited

# run through auditlite so the execution is fingerprinted
auditlite run --input data.csv --output table1.csv --seed numpy=42 \
    -- python analysis.py

# tie manuscript numbers to the run that produced them
auditlite claim table1 table1.csv

# before submission (or in CI)
auditlite check
auditlite report        # writes audit/AUDIT.md

Same API from Python:

import auditlite

trail = auditlite.init(project="lung-registry-survival")
trail.provenance("analysis.py", origin="ai_modified", model="claude-sonnet-5")
trail.decision("model", "logistic regression", "binary outcome, prespecified")
trail.run(["python", "analysis.py"], inputs=["data.csv"], outputs=["table1.csv"])
trail.claim("table1", "table1.csv")

for result in auditlite.run_checks(trail):
    print(result.check, result.status, result.detail)

The trail format

One JSON object per line. Every record carries the SHA-256 of the previous record, so any later edit breaks the chain from that point forward:

{"seq": 4, "ts": "2026-07-11T09:12:03+00:00", "kind": "run", "actor": "human",
 "body": {"command": ["python", "analysis.py"], "scripts": [...], "inputs": [...],
          "outputs": [...], "seeds": {"numpy": 42}, "exit_code": 0,
          "environment": {"python": "3.12.4", "platform": "...", "packages": {...}}},
 "prev": "9f2c…", "hash": "a41b…"}

Plain text, local-first, no server, no account. Commit audit/ next to the analysis and the trail travels with the project.

Siblings

• recoverlite checks whether a planned method can recover its target.
• assesslite checks whether a conclusion survives plausible assumptions.
• auditlite checks whether the entire analytic chain can be reconstructed and defended.

License

Apache-2.0. © Heidi Helena Andersen.