llm_router →

Provider-agnostic LLM router with YAML routing, auto-failover, cost telemetry, and multi-model support.
One API. Any provider. Zero vendor lock-in.

llm_router is a small Python package (~500 LOC core) that gives you a uniform call_tool() interface across Anthropic, DeepInfra, OpenRouter (30+ models), and more. Define routing rules in YAML — change providers without touching application code. In production since May 2026 powering second-brain-instance.

Why llm_router?

• 🔀 Multi-provider by default — Anthropic, DeepInfra, OpenRouter. Add more via config.
• 🛡️ Auto-failover — provider down? Router switches to fallback automatically (circuit breaker)
• 💰 Cost-aware — route to cheapest available model. Daily budget guard.
• 📊 Telemetry built-in — every call logged (provider, model, tokens, cost, latency)
• 📝 YAML-only config — change routing rules without touching code
• 🇵🇱 Polish-optimized — routing rules aware of PL language quality per model
• 🧩 Drop-in — replace direct anthropic SDK calls with call_tool(use_case=..., ...)

Quick Start

pip install llm-router

from llm_router import call_tool, set_routing_config

# One-time setup (put in your app's __init__)
set_routing_config('config/llm_routing.yaml')

# Use anywhere — same API regardless of provider
result, usage = call_tool(
    use_case='myapp.feature_x',
    system='You are a helpful assistant.',
    user='Summarize this text: ...',
    schema={'name': 'summarize', 'input_schema': {...}},
)
print(f'${usage.cost_usd:.4f} via {usage.provider}/{usage.model}')

Full walkthrough: docs/INTEGRATION.md (5 min).

Configuration

# config/llm_routing.yaml
providers:
  deepinfra:
    api_key_env: DEEPINFRA_API_KEY
    default_model: Qwen/Qwen3.5-397B-A17B
  openrouter:
    api_key_env: OPENROUTER_API_KEY
    base_url: https://openrouter.ai/api/v1
  anthropic:
    api_key_env: ANTHROPIC_API_KEY

use_cases:
  myapp.feature_x:
    primary: { provider: deepinfra, model: Qwen/Qwen3.5-397B-A17B }
    fallback:
      - { provider: openrouter, model: qwen/qwen3.5-397b-a17b }
      - { provider: anthropic, model: claude-haiku-4-5-20251001 }
    cost_stop_per_call_usd: 0.50

Status

✅ In production — routing all LLM calls for second-brain-instance since May 2026.

• 30+ calls/day across 11 perspective types
• DeepInfra Qwen primary, OpenRouter fallback, Anthropic last resort
• Provider error rate monitored with auto-failover (R4 — in progress)

Documentation

• docs/INTEGRATION.md — 5-min quick-start
• docs/ARCHITECTURE.md — design, provider protocol, error handling
• docs/IMPLEMENTATION_PLAN.md — roadmap, tasks

Decision history:

• docs/research/llm-cost-optimization-analysis.md
• docs/research/open-weight-llms.md

Strategy:

• docs/router-strategy-analysis.md — full strategic analysis: requirements, market, RICE prioritization, cost, cross-domain expansion

License

MIT © 2026 Eryk Orłowski

Built for second-brain-instance — now open to everyone.