ai-trace-auditor 0.17.0

AI compliance auditor: EU AI Act, NIST AI RMF, ISO 42001, SOC 2. Every requirement verified against primary legal text with exact clause citations.

AI Trace Auditor

The EU AI Act takes effect August 2, 2026. Your AI system needs compliance evidence. Your observability tools collect traces. Your GRC platform manages policies. Nothing translates traces into compliance evidence. This tool does.

Open-source CLI that audits LLM traces against EU AI Act Articles 11, 12, 13, and 25, plus NIST AI RMF and GDPR Article 30. 301 tests. Zero LLM dependencies. Runs locally.

The Problem

Companies have traces from Langfuse, Arize, or OpenTelemetry, but no automated way to answer: "Do our traces satisfy the regulatory requirements?" 67% of AI teams discover quality regressions from user complaints despite having tracing infrastructure. The gap isn't data collection; it's interpretation.

Consultants charge $30K-$500K per compliance engagement. CEPS estimates EUR 29K per AI model for Annex IV documentation alone. This tool automates the gap analysis for free.

What It Covers

Regulation	Scope
EU AI Act Article 11	Technical documentation (Annex IV). Auto-generates 60-70% of required sections from code scanning
EU AI Act Article 12	Record-keeping. Audits trace data against 15+ discrete requirements
EU AI Act Article 13	Transparency (provider to deployer). Data flow mapping with GDPR role classification
EU AI Act Article 25	Value chain accountability. Multi-agent DAG auditing with per-agent penalty propagation
NIST AI RMF	GOVERN, MAP, MEASURE, MANAGE subcategories (~15 requirements)
GDPR Article 30	Records of Processing Activities. Auto-generated from detected data flows

Multi-Agent Compliance (v0.14.0)

Automatically audits multi-agent systems (LangGraph, CrewAI, AutoGen, Google ADK):

• Reconstructs execution DAGs from parent-child span relationships
• Per-agent compliance scores with bottom-up penalty propagation
• Article 25 "value chain accountability" checks
• Liability shift detection (deployer becoming provider)
• Mermaid DAG visualizations

No other open-source tool does multi-agent compliance auditing.

Supported Trace Formats

Format	Source
OTel OTLP JSON	OpenTelemetry GenAI semantic conventions
Langfuse JSON	Langfuse trace exports
Claude Code	~/.claude/projects/ conversation traces
Raw JSONL	Any provider's API logs

Auto-detected. Use --format to override.

Install

pip install ai-trace-auditor

Quick Start

# Audit traces against EU AI Act
aitrace audit traces.json -r "EU AI Act" -o report.md

# Full compliance package: Articles 11 + 12 + 13 + GDPR in one run
aitrace scan ./my-ai-project/ --traces traces.json

# Generate Annex IV technical documentation from code
aitrace docs ./my-ai-project/

# Map data flows for Article 13 + GDPR Article 30
aitrace flow ./my-ai-project/

# Multi-agent audit with DAG visualization
aitrace audit multi_agent_traces.json --show-dag

Audit a GitHub repository

aitrace audit-repo https://github.com/owner/repo

The auditor clones the repo (shallowly, 50MB cap, 30s timeout), discovers trace artifacts (OTEL JSON, Langfuse exports, chat JSONL), and runs the governance-doc manifest (MODEL_CARD.md, retention configs, Article 50 disclosure language, ISO/SOC 2 policy docs). The web dashboard also accepts a repo URL at /audit.

Example Output

Real output from auditing 1,522 Claude Code spans:

Overall Compliance Score: 79.3%

| Status    | Count |
|-----------|-------|
| Satisfied |    10 |
| Partial   |     5 |
| Missing   |     3 |

Top gaps:
  1. Not logging: Temperature parameter controlling output randomness
  2. Not logging: Maximum token limit for output generation
  3. Incomplete: Output responses generated by the AI model (31.9% coverage)
  4. Incomplete: Input prompts/messages (4.7% coverage)
  5. Not logging: Operation latency in milliseconds

GitHub Action

- name: Audit AI traces
  uses: BipinRimal314/ai-trace-auditor@v0.14.0
  with:
    path: traces/exported.json
    regulation: "EU AI Act"
    output: compliance-report.md
    fail-on-gaps: "true"

Exit code 0 = all requirements satisfied, 1 = gaps found. CI-friendly.

Library API

from ai_trace_auditor.ingest import ingest_file
from ai_trace_auditor.analysis.engine import ComplianceAnalyzer
from ai_trace_auditor.regulations.registry import RequirementRegistry

traces = ingest_file(Path("traces.json"))
registry = RequirementRegistry()
registry.load()

report = ComplianceAnalyzer(registry).analyze(
    traces=traces,
    regulations=["EU AI Act"],
)

print(f"Score: {report.overall_score:.1%}")

Architecture

ai-trace-auditor/
├── src/ai_trace_auditor/
│   ├── cli.py              # 7 commands: audit, docs, flow, scan, ingest, requirements, health
│   ├── ingest/             # Trace ingestion (OTel, Langfuse, Claude Code, raw JSONL)
│   ├── analysis/           # Gap analysis engine + multi-agent DAG auditing
│   ├── models/             # Pydantic v2 data models
│   ├── regulations/        # YAML requirement definitions (extensible)
│   ├── docs/               # Article 11 Annex IV generator
│   ├── flow/               # Article 13 data flow mapper + GDPR RoPA
│   ├── scan/               # Full compliance package runner
│   ├── evidence/           # Auditor-ready evidence pack generator
│   ├── reports/            # Markdown, JSON, PDF report generation
│   ├── scanner/            # Code scanner (AI framework detection)
│   └── guide_linter/       # Lints compliance guides for accuracy
├── requirements/           # YAML regulatory requirement definitions
│   ├── eu_ai_act/          # Articles 12, 19
│   └── nist_ai_rmf/       # GOVERN, MAP, MEASURE, MANAGE
└── tests/                  # 301 tests

No dependency on any LLM framework. Intentionally framework-agnostic.

Competitive Landscape

Tool	What It Does	How We Differ
Langfuse / Arize	Collect and visualize traces	We interpret traces against regulations
Credo AI ($41M raised)	Enterprise AI governance platform	Free, open-source, runs locally
OneTrust / Vanta	GRC policy management	We ingest AI-specific traces, not generic policy docs
Holistic AI	AI risk management (consulting)	CLI + CI integration, no sales call required

The competition is consultants, not software.

Disclaimer

This tool provides automated compliance analysis. It is not legal advice. Risk classification under Annex III requires legal review. Consult qualified counsel for compliance decisions.

License

Apache 2.0