ai-trace-auditor 0.14.0

EU AI Act compliance suite: Articles 11, 12, 13, 25 + GDPR Article 30. Multi-agent DAG auditing with per-agent compliance scoring.

AI Trace Auditor

Audit LLM traces against regulatory compliance requirements. Open-source CLI that sits between your observability stack (Langfuse, Arize, OTel) and regulatory frameworks (EU AI Act, NIST AI RMF).

Your observability tools collect traces. Your GRC platform manages policies. Nothing translates traces into compliance evidence. This tool does.

Install

pip install ai-trace-auditor

Or from source:

git clone https://github.com/BipinRimal314/ai-trace-auditor.git
cd ai-trace-auditor
pip install -e .

Quick Start

# Audit traces against all regulations
aitrace audit traces.json

# Audit against a specific regulation
aitrace audit traces.json -r "EU AI Act" -o report.md

# Audit your Claude Code conversation traces
aitrace audit ~/.claude/projects/*/session-id.jsonl

# Inspect what requirements exist
aitrace requirements --show EU-AIA-12.1

# Just ingest and summarize traces
aitrace ingest traces.json --summary

GitHub Action

Add compliance checks to your CI pipeline:

- name: Audit AI traces
  uses: BipinRimal314/ai-trace-auditor@v0.13.0
  with:
    path: traces/exported.json
    regulation: "EU AI Act"
    output: compliance-report.md
    fail-on-gaps: "true"

The action fails if compliance gaps are found. Set fail-on-gaps: "false" to report without blocking.

What It Checks

EU AI Act Article 12 (Record-Keeping):

• Event timestamps, operation identification
• Risk situation logging (errors, failure modes)
• Model version tracking for post-market monitoring
• Resource consumption (tokens, latency)
• Content recording (opt-in)
• Tool/function call audit trails
• Trace linkage for multi-step operations

EU AI Act Article 25 (Value Chain Accountability) -- NEW in v0.13.0:

• Agent identity traceability across multi-agent systems
• Delegation chain documentation (who delegated what to whom)
• MCP boundary transparency (cross-boundary tool calls)
• Substantial modification detection (deployer-to-provider liability shift)
• Bottom-up penalty propagation: downstream agent failures cascade to upstream delegators
• Per-agent compliance scores with system-level aggregation

NIST AI RMF:

• Production monitoring (MEASURE 2.4)
• Transparency documentation (MEASURE 2.8)
• Model explainability (MEASURE 2.9)
• Risk tracking (MEASURE 3.1)
• Post-deployment monitoring (MANAGE 4.1)
• Incident communication (MANAGE 4.3)

Supported Trace Formats

Format	Source
OTel OTLP JSON	OpenTelemetry GenAI semantic conventions
Langfuse JSON	Langfuse trace exports
Claude Code	~/.claude/projects/ conversation traces
Raw JSONL	Any provider's API logs

Auto-detected. Use --format to override.

Multi-Agent Support (v0.13.0)

Automatically detects multi-agent traces from LangGraph, CrewAI, AutoGen, and Google ADK. When multiple agents are detected:

• Reconstructs the execution DAG from parent-child span relationships
• Computes per-agent compliance scores with bottom-up penalty propagation
• Checks Article 25 "value chain accountability" requirements
• Detects liability shifts when deployers may become providers
• Generates Mermaid DAG visualizations with --show-dag

# Audit a multi-agent trace with DAG visualization
aitrace audit multi_agent_traces.json --show-dag

# Output includes per-agent scores:
# Per-Agent Compliance Scores
# | orchestrator-1 | 33.0% |  (red: penalized for downstream failures)
# | researcher-1   | 66.4% |  (amber: own gaps + some coverage)
# | writer-1       | 66.4% |  (amber: own gaps + some coverage)

Single-agent traces continue to work identically. Article 25 requirements are only checked when multiple agents are detected.

Example Output

Real output from auditing 1,522 Claude Code spans:

Overall Compliance Score: 79.3%

| Status    | Count |
|-----------|-------|
| Satisfied |    10 |
| Partial   |     5 |
| Missing   |     3 |

Top gaps:
  1. Not logging: Temperature parameter controlling output randomness
  2. Not logging: Maximum token limit for output generation
  3. Incomplete: Output responses generated by the AI model (31.9% coverage)
  4. Incomplete: Input prompts/messages (4.7% coverage)
  5. Not logging: Operation latency in milliseconds

CI Integration

Exit code 0 = all satisfied, 1 = gaps found:

aitrace audit traces.json -r "EU AI Act" || echo "Compliance gaps detected"

Library API

Use programmatically in your own tools:

from ai_trace_auditor.ingest import ingest_file
from ai_trace_auditor.analysis.engine import ComplianceAnalyzer
from ai_trace_auditor.regulations.registry import RequirementRegistry

traces = ingest_file(Path("traces.json"))

registry = RequirementRegistry()
registry.load()

report = ComplianceAnalyzer(registry).analyze(
    traces=traces,
    regulations=["EU AI Act"],
)

print(f"Score: {report.overall_score:.1%}")
for result in report.requirement_results:
    if result.gaps:
        print(f"  {result.requirement.id}: {result.gaps[0].recommendation}")

Disclaimer

This tool provides automated compliance assessments based on its interpretation of regulatory requirements. It is not legal advice. Consult qualified legal counsel for compliance decisions.

License

Apache 2.0