fableforge-trajectory-distiller 0.1.0

Distill agent traces into training datasets across multiple formats

Trajectory Distiller

Convert agent traces from multiple formats into training datasets for fine-tuning.

Installation

pip install trajectory-distiller

Supported Input Formats

Format	Description
glint	Session-based format with turns array
armand0e	Conversation-based format with tool_calls
vfable	Trajectory-based format with tool_use
opencoven	Source/target pair format
victor	Prompt/response pair format

Supported Output Formats

Format	Description
openai_chat	OpenAI chat completion format
alpaca	Alpaca instruction format
sharegpt	ShareGPT conversation format
conversation	General conversation format

Quick Start

Distill Traces

# Convert glint traces to OpenAI chat format (auto-detected)
distill input.jsonl --format openai_chat --output train.jsonl

# Convert armand0e format explicitly
distill input.jsonl --input-format armand0e --format sharegpt -o train.jsonl

# Convert to alpaca format
distill input.jsonl --format alpaca -o alpaca_train.jsonl

Filter Traces

# Filter to records using specific tools
distill filter traces.jsonl --tool bash --tool edit

# Filter by error rate and quality
distill filter traces.jsonl --min-errors 0.1 --min-quality 0.5

# Filter by session length
distill filter traces.jsonl --min-turns 5 --max-turns 50

# Combine filters and save
distill filter traces.jsonl --tool bash --min-quality 0.3 -o filtered.jsonl

Split Dataset

# Split into 95/5 train/val
distill split traces.jsonl --train-ratio 0.95 --val-ratio 0.05

# Stratify by tool distribution
distill split traces.jsonl --stratify-by tool --output-dir splits/

# Split with test set
distill split traces.jsonl --train-ratio 0.8 --val-ratio 0.1 --test-ratio 0.1

Fable5 Dataset Usage

# Glint dataset
distill glint_traces.jsonl --format openai_chat -o glint_openai.jsonl

# armand0e dataset
distill armand0e_data.jsonl --input-format armand0e --format alpaca -o armand0e_alpaca.jsonl

# vfable dataset
distill vfable_traces.jsonl --input-format vfable --format sharegpt -o vfable_sharegpt.jsonl

# opencoven dataset
distill opencoven_pairs.jsonl --input-format opencoven --format openai_chat -o opencoven_openai.jsonl

# victor dataset
distill victor_pairs.jsonl --input-format victor --format conversation -o victor_conv.jsonl

Programming API

from trajectory_distiller import Distiller, FormatConverter, TraceFilter, DataSplitter

# Distill traces
distiller = Distiller()
records = distiller.distill("traces.jsonl", output_format="openai_chat")

# Filter traces
trace_filter = TraceFilter()
filtered = trace_filter.filter_by_tool(records, tools=["bash", "edit"])
filtered = trace_filter.filter_by_quality(filtered, min_quality_score=0.5)

# Convert formats
converter = FormatConverter()
alpaca_records = converter.to_alpaca(records)
sharegpt_records = converter.to_sharegpt(records)

# Split data
splitter = DataSplitter()
splits = splitter.split(records, train_ratio=0.95, stratify_by="tool")
splits.save("output/")
print(splits.stats())

License

MIT

Ecosystem

Part of the FableForge ecosystem — 21 open-source projects built from 210K real agent traces:

Project	Description
Anvil	Self-verified coding agent
VerifyLoop	Plan→Execute→Verify→Recover framework
ErrorRecovery	Self-healing middleware (3,725 error patterns)
FableForge-14B	The fine-tuned 14B model (4-stage training)
ShellWhisperer	1.5B edge agent (phone/RPi, 50ms)
ReasonCritic	Verification model (130 benchmark tasks)
TraceCompiler	Compile traces → LoRA skills
AgentRuntime	Persistent agent daemon (systemd for AI)
AgentSwarm	Multi-agent from real trace transitions
AgentTelemetry	Datadog for agents (token tracking, costs)
BenchAgent	HumanEval for tool-use (107 tasks)
AgentDev	VSCode extension with verification
TraceViz	Trace replay visualizer (Next.js)
AgentSkills	npm for agent behaviors
AgentCurriculum	5-stage progressive training
AgentFuzzer	Adversarial testing for agents
AgentConstitution	Safety guardrails from traces
CostOptimizer	Token cost reduction (50-80%)
AgentProfiler	Behavioral fingerprinting
TrajectoryDistiller	Trace→training data pipeline
Fable5-Dataset	HuggingFace dataset release