TraceCompiler

Compile agent traces into distilled LoRA weights for specific coding behaviors.

Overview

TraceCompiler parses JSONL traces from multiple agent formats (Glint, armand0e, v-Fable), extracts named skill examples (debug, edit, verify, recover, plan), and distills them into LoRA adapter weights that can be loaded onto a base language model.

Architecture

JSONL Traces → Parser → SkillExtractor → Distiller → LoRA Adapter
                                                       ↓
                                                  Evaluator → Report

Supported Trace Formats

Glint Format

{
  "uid": "glint-001",
  "source_file": "main.py",
  "session": "session-abc",
  "model": "claude-3-opus",
  "context": "Fix the bug on line 10",
  "cot": "Let me analyze this error...",
  "output_type": "text",
  "output": "The issue is a missing import.",
  "completion": "",
  "origin": "glint"
}

armand0e Format

{
  "type": "ai",
  "message": {
    "content": [
      {"type": "thinking", "thinking": "Analyzing the code..."},
      {"type": "text", "text": "Here's the fix."},
      {"type": "tool_use", "id": "call_1", "name": "bash", "input": {"command": "pytest"}}
    ],
    "model": "claude-3-opus"
  },
  "parentUuid": "msg-parent",
  "uuid": "msg-child"
}

v-Fable Format

Same as Glint but with "origin": "v-Fable".

Installation

pip install -e .

For development:

pip install -e ".[dev]"

Usage

Parse Traces

# Parse a trace file and display summary
trace-compiler parse traces.jsonl

# Parse and save to file
trace-compiler parse traces.jsonl -o parsed.json

# Verbose output
trace-compiler parse traces.jsonl -v

# Specify format explicitly
trace-compiler parse traces.jsonl --format glint

Extract Skills

# Extract debug skill examples
trace-compiler extract --skill debug traces.jsonl

# Extract with custom confidence threshold
trace-compiler extract --skill debug --min-confidence 0.7 traces.jsonl

# Extract and save examples
trace-compiler extract --skill debug -o examples.json traces.jsonl

Available skills:

• debug — Error recovery traces
• edit — Code edit tool calls
• verify — Verification reasoning
• recover — Error → recovery patterns
• plan — Planning reasoning

Compile into LoRA Adapter

# Compile using default settings (Qwen2.5-Coder-1.5B)
trace-compiler compile --skill debug traces.jsonl

# Use a specific model alias
trace-compiler compile --skill debug --model qwen3-7b traces.jsonl

# Use a training config file
trace-compiler compile --skill debug --config configs/debug.yaml traces.jsonl

# Specify output directory
trace-compiler compile --skill debug -o ./my-adapters traces.jsonl

Model aliases:

• qwen3-1.5b → Qwen/Qwen2.5-Coder-1.5B
• qwen3-7b → Qwen/Qwen2.5-Coder-7B
• codellama-7b → codellama/CodeLlama-7b-hf
• mistral-7b → mistralai/Mistral-7B-v0.1

Or pass a full HuggingFace model ID.

Evaluate Adapted Model

# Evaluate a debug skill adapter
trace-compiler evaluate --skill debug --adapter ./output/debug traces.jsonl

# Specify model and save report
trace-compiler evaluate --skill debug --model qwen3-1.5b \
  --adapter ./output/debug -o report.json traces.jsonl

Inspect Trace Files

# Quick format detection
trace-compiler inspect traces.jsonl

# Verbose inspection with content analysis
trace-compiler inspect traces.jsonl -v

Training Configuration

Each skill type has a default config in configs/. You can customize:

# configs/debug.yaml
skill_type: debug

training:
  model_name: Qwen/Qwen2.5-Coder-1.5B
  lora_r: 16
  lora_alpha: 32
  lora_dropout: 0.05
  learning_rate: 2.0e-4
  num_epochs: 3
  batch_size: 4
  gradient_accumulation_steps: 4
  max_seq_length: 4096
  warmup_steps: 10
  weight_decay: 0.01

LoRA Parameters

Parameter	Default	Description
lora_r	16	LoRA rank (higher = more capacity, slower)
lora_alpha	32	LoRA scaling factor (typically 2x rank)
lora_dropout	0.05	Dropout probability for LoRA layers
max_seq_length	4096	Maximum sequence length for training

Target Modules

Default target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Skill Extraction Details

DEBUG Skill

Detects error recovery patterns — traces containing error messages, exceptions, tracebacks followed by investigation and fixes.

EDIT Skill

Identifies code modification traces — tool calls to edit/replace/write tools, or content containing edit instructions.

VERIFY Skill

Extracts verification reasoning — traces where the model runs tests, checks outputs, or explicitly verifies changes.

RECOVER Skill

Finds error-to-recovery sequences — an error message followed by a fix attempt within a short window.

PLAN Skill

Detects planning patterns — traces where the model explicitly outlines a strategy before implementation.

Programmatic Usage

from trace_compiler import TraceParser, SkillExtractor, SkillType, Distiller, TrainingConfig

# Parse traces
parser = TraceParser()
records = parser.parse_file("traces.jsonl")

# Extract skills
extractor = SkillExtractor(min_confidence=0.6)
debug_examples = extractor.extract(records, skill_type=SkillType.DEBUG)

# Configure and train
config = TrainingConfig.from_yaml("configs/debug.yaml")
distiller = Distiller(config=config)
adapter_path = distiller.train(debug_examples, SkillType.DEBUG)
print(f"Adapter saved to: {adapter_path}")

Evaluation

The evaluator runs benchmark prompts against the base model and the adapted model, scoring responses against skill-specific criteria:

• DEBUG: Identifies errors, suggests fixes, explains root causes
• EDIT: Makes minimal, correct edits preserving existing behavior
• VERIFY: Suggests systematic verification with edge cases
• RECOVER: Diagnoses failures and provides recovery steps
• PLAN: Produces structured, prioritized plans

Scores are computed using keyword matching against criteria. A positive improvement delta indicates the adapter improved over the base model.

Requirements

• Python 3.10+
• PyTorch 2.1+
• CUDA-capable GPU recommended for training
• 8GB+ VRAM for Qwen2.5-Coder-1.5B with LoRA
• 16GB+ VRAM for Qwen2.5-Coder-7B with LoRA

License

MIT

Ecosystem

Part of the FableForge ecosystem — 21 open-source projects built from 210K real agent traces:

Project	Description
Anvil	Self-verified coding agent
VerifyLoop	Plan→Execute→Verify→Recover framework
ErrorRecovery	Self-healing middleware (3,725 error patterns)
FableForge-14B	The fine-tuned 14B model (4-stage training)
ShellWhisperer	1.5B edge agent (phone/RPi, 50ms)
ReasonCritic	Verification model (130 benchmark tasks)
TraceCompiler	Compile traces → LoRA skills
AgentRuntime	Persistent agent daemon (systemd for AI)
AgentSwarm	Multi-agent from real trace transitions
AgentTelemetry	Datadog for agents (token tracking, costs)
BenchAgent	HumanEval for tool-use (107 tasks)
AgentDev	VSCode extension with verification
TraceViz	Trace replay visualizer (Next.js)
AgentSkills	npm for agent behaviors
AgentCurriculum	5-stage progressive training
AgentFuzzer	Adversarial testing for agents
AgentConstitution	Safety guardrails from traces
CostOptimizer	Token cost reduction (50-80%)
AgentProfiler	Behavioral fingerprinting
TrajectoryDistiller	Trace→training data pipeline
Fable5-Dataset	HuggingFace dataset release