RouteSmith

The smart router for AI coding tools. Backed by contextual bandit research — see Research for measured results.

pip install "routesmith-llm[proxy]"
routesmith quickstart        # detects your provider keys, builds a matching model pool
routesmith run claude        # or: routesmith run codex / routesmith run opencode

That's it — Claude Code (or Codex, or OpenCode) now runs exactly as normal, except every request is silently routed to the best model for it. No tool config to edit, no model picker, no server terminal to babysit.

RouteSmith sits between your AI coding tool and the LLM. It routes every request to the best model for that specific task — cheap models for simple edits, frontier models for complex refactors. You never think about model IDs again.

	OpenRouter Auto	RouteSmith
Learns from your traffic	❌	✅ Online bandit learning
Self-hosted / no data leak	❌	✅ Open source, self-hosted
Custom model pool	❌	✅ Any model, any provider
Custom rewards & policies	❌	✅ Per-role, configurable reward fns
Decision audit log	❌	✅ Full routing decision trace
Budget caps (daily/hourly)	❌	✅ Monthly, per-request, per-project
Conversation stickiness	✅	✅

v0.9.0: routesmith run <tool> one-command integration, provider-aware model catalogs, routesmith models refresh, persistent conversation stickiness, routesmith connect --verify. See CHANGELOG.md.

Who it's for

💰 You pay for API access. Cut your bill. Your Claude Code or Codex session burns through tokens on every turn, including subagent calls. RouteSmith sends typos and formatting to gpt-4o-mini, saves Claude Opus for architecture decisions. Paper experiments measured 45% cost savings at 71% accuracy on 5-model routing — see Research.

🆓 You use free models. Get better answers. Free models are good individually — but none is great at everything. RouteSmith orchestrates them: hard problems get the strongest free model, easy ones get the fastest, and weak answers cascade to second opinions.

from routesmith import RouteSmith

# Free models — zero cost, smart routing
rs = RouteSmith.with_free_models()

# Or bring your own models
rs = RouteSmith()
rs.register_model("gpt-4o-mini", cost_per_1k_input=0.15, cost_per_1k_output=0.60, quality_score=0.85)
rs.register_model("claude-sonnet-4", cost_per_1k_input=3.0, cost_per_1k_output=15.0, quality_score=0.92)

response = rs.completion(messages=[{"role": "user", "content": "Explain recursion"}])

AI coding tools

The fastest path for any tool is routesmith run <tool> (see Quick start). For manual setup or CI, routesmith connect <tool> prints the exact config, and routesmith connect <tool> --verify confirms requests are actually being routed (not just reachable):

Tool	routesmith connect <tool>
Claude Code	claude-code — sets ANTHROPIC_BASE_URL (native /v1/messages, full tool-use support)
Codex	codex — sets OPENAI_BASE_URL + wire_api: chat
OpenCode	opencode
OpenClaw	openclaw
pi	pi
Hermes	hermes
OpenAI SDK	openai-sdk
Anthropic SDK	anthropic-sdk

Integration guides →

Why RouteSmith

	Raw OpenRouter	Manual routing	RouteSmith
Picks model per query	❌	😓 You do it	✅ Automatic
Cascades when answer is weak¹	❌	❌	✅
Caches repetitive queries	❌	❌	✅
Enforces budget limits	❌	❌	✅
Tracks costs per model	✅	❌	✅
Works with 100+ models	✅	❌	✅
Zero-config start	✅	❌	✅
Learns from feedback	❌	❌	✅

¹ Today all strategies select a single model; cascade execution lands in Phase 2 (see ROADMAP.md).

Features

Intelligent Routing

• 7 predictor types: LinUCB, LinTS, NeuralUCB, REINFORCE, WarmStart LinUCB, Adaptive (random forest), Embedding
• 35-dimensional feature space: query type classification, difficulty estimation, model metadata
• Online learning: bandits improve from the first query onward — no pretraining labels needed
• Multi-model routing: scales to $K$ arms naturally (validated on 5-model deployments)

Enterprise

• Provisioned throughput support: prioritize pre-paid capacity, overflow to on-demand
• Compliance routing: tag-based filtering (HIPAA, SOC2, PCI)
• Budget enforcement: FAIL, FALLBACK, QUEUE behaviors
• Multi-project isolation: per-project cost allocation and stats

Production

• Semantic cache: embedding-based dedup, configurable similarity
• Framework adapters: LangChain, DSPy, CrewAI, AutoGen, Anthropic, OpenClaw
• OpenAI-compatible proxy: works with any tool, zero code changes
• Observability: Prometheus metrics, structured logging, cost tracking, dashboard TUI
• Resilience: circuit breakers, retry with backoff, health checks, Docker

Research

RouteSmith is backed by a research paper evaluating contextual bandit routing:

• LinTS-35d achieves 46% cost reduction with APGR=0.593 on MMLU
• LinUCB-35d achieves APGR=1.126 by selective strong-arm routing
• 5-arm routing: 45% cost savings across GPT-4o, Claude-Sonnet-4.5, Qwen-Plus, MiniMax-M1, DeepSeek-V3
• Zero pretraining labels — learns from ~100 queries vs. 55K+ required by supervised routers
• Sub-millisecond routing latency (<0.5ms P99)

Paper: paper/main.pdf | Compile with: cd paper && tectonic main.tex

Framework integrations

# Anthropic SDK
from routesmith.integrations.anthropic import RouteSmithAnthropic
client = RouteSmithAnthropic.with_openrouter_models()

# LangChain
from routesmith.integrations.langchain import ChatRouteSmith
llm = ChatRouteSmith.with_openai_models()

# DSPy
from routesmith.integrations.dspy import RouteSmithLM
lm = RouteSmithLM()

# CrewAI
from routesmith.integrations.crewai import routesmith_crewai_chat_model
llm = routesmith_crewai_chat_model()

# AutoGen
from routesmith.integrations.autogen import routesmith_autogen_agents
assistant, user = routesmith_autogen_agents()

Quick start

# Interactive setup: browse OpenRouter catalog, pick models
routesmith init

# Start the proxy
routesmith serve

# Check stats
routesmith stats

# View routing decisions
routesmith audit

# Manage per-role routing policies
routesmith roles list
routesmith roles set --role coder --model-pool gpt-4o-mini gpt-4o
routesmith roles unset --role coder

# Python API
from routesmith import RouteSmith

rs = RouteSmith.with_free_models()
response = rs.completion(messages=[{"role": "user", "content": "Hello!"}])
print(response.choices[0].message.content)

# Learn from user feedback
rs.record_outcome(response._routesmith_request_id, score=0.9)

Examples

File	Requires	Description
examples/quickstart_python.py	—	Register models, completion, stats, feedback
examples/quickstart_proxy.sh	routesmith[proxy]	Proxy via CLI: init → serve → curl
examples/multi_agent_roles.py	—	Per-role routing (planner/coder/summarizer)
examples/langgraph_agents.py	langchain_core	2-node LangGraph with per-role ChatRouteSmith
examples/crewai_crew.py	crewai	2-agent CrewAI crew with shared RouteSmith
examples/autogen_pair.py	autogen	AutoGen agent pair via proxy
examples/dspy_pipeline.py	dspy	DSPy Predict with RouteSmithLM
examples/openai_agents_sdk.py	openai	OpenAI SDK pointed at proxy
examples/pydantic_ai_agent.py	pydantic_ai	Pydantic AI agent via proxy
examples/llamaindex_engine.py	llama_index	LlamaIndex OpenAILike via proxy

Documentation

• CHANGELOG.md
• CONTRIBUTING.md
• Integration guides
• CLI reference
• Concepts

Installation

# Proxy + interactive setup (recommended)
pip install "routesmith-llm[proxy]"

# Core Python API only
pip install routesmith-llm

# With specific integrations
pip install "routesmith-llm[langchain]"
pip install "routesmith-llm[anthropic]"
pip install "routesmith-llm[cache]"
pip install "routesmith-llm[all]"

Requires Python 3.10+. Set OPENROUTER_API_KEY to use OpenRouter models.

License

MIT — see LICENSE