fableforge-cost-optimizer 0.1.0

Analyze token waste and optimize LLM routing for cost savings

Cost Optimizer

Analyze token waste in LLM agent traces and optimize cost through intelligent model routing.

Installation

pip install cost-optimizer

Quick Start

Analyze Traces

# Analyze a trace file for waste
costopt analyze trace.jsonl

# Specify model for cost calculation
costopt analyze trace.jsonl --model gpt-4o

# Save results
costopt analyze trace.jsonl -o results.json

Estimate Costs

# Estimate cost for 1M tokens on Claude 3.5 Sonnet
costopt estimate --model claude-3-5-sonnet-20241022 --tokens 1000000

# Compare costs across all models
costopt estimate --model claude-3-5-sonnet-20241022 --tokens 1000000 --compare

Optimize

# Get optimization recommendations
costopt optimize trace.jsonl

Programming API

from cost_optimizer import TokenAnalyzer, CostOptimizer, ModelRouter, PricingData

# Analyze traces
analyzer = TokenAnalyzer(default_model="claude-3-5-sonnet-20241022")
report = analyzer.analyze_trace("trace.jsonl")
print(f"Waste: {report.total_waste_tokens} tokens")

# Get optimization recommendations
optimizer = CostOptimizer()
optimizations = optimizer.optimize(report)
for opt in optimizations:
    print(f"{opt.strategy}: saves ${opt.estimated_savings_usd:.2f}")

# Route to cheaper models
router = ModelRouter()
model = router.route("simple formatting task")  # → claude-3-5-haiku-20241022
assessment = router.assess("design a distributed system architecture")
print(f"Complexity: {assessment.complexity_score}, Model: {assessment.recommended_model}")

# Calculate costs
cost = PricingData.calculate_cost(1000000, "claude-3-5-sonnet-20241022")

Supported Models

Model	Tier	Input/1M tokens	Output/1M tokens
Claude 3.5 Haiku	mini	$0.80	$4.00
GPT-4o Mini	mini	$0.15	$0.60
Qwen3 Coder	mini	$0.50	$1.50
Claude 3.5 Sonnet	standard	$3.00	$15.00
GPT-4o	standard	$2.50	$10.00
GPT-4 Turbo	premium	$10.00	$30.00
GPT-4	premium	$30.00	$60.00
Claude 3 Opus	flagship	$15.00	$75.00

License

MIT

Ecosystem

Part of the FableForge ecosystem — 21 open-source projects built from 210K real agent traces:

Project	Description
Anvil	Self-verified coding agent
VerifyLoop	Plan→Execute→Verify→Recover framework
ErrorRecovery	Self-healing middleware (3,725 error patterns)
FableForge-14B	The fine-tuned 14B model (4-stage training)
ShellWhisperer	1.5B edge agent (phone/RPi, 50ms)
ReasonCritic	Verification model (130 benchmark tasks)
TraceCompiler	Compile traces → LoRA skills
AgentRuntime	Persistent agent daemon (systemd for AI)
AgentSwarm	Multi-agent from real trace transitions
AgentTelemetry	Datadog for agents (token tracking, costs)
BenchAgent	HumanEval for tool-use (107 tasks)
AgentDev	VSCode extension with verification
TraceViz	Trace replay visualizer (Next.js)
AgentSkills	npm for agent behaviors
AgentCurriculum	5-stage progressive training
AgentFuzzer	Adversarial testing for agents
AgentConstitution	Safety guardrails from traces
CostOptimizer	Token cost reduction (50-80%)
AgentProfiler	Behavioral fingerprinting
TrajectoryDistiller	Trace→training data pipeline
Fable5-Dataset	HuggingFace dataset release