llm-router
Stop spending your Claude Pro/Max quota on questions a free model can answer.
llm-router hooks into your coding tool's own lifecycle, reads each prompt before the model does, and drafts an answer on a free or local model first. No API keys needed on a Claude subscription — routing runs through MCP tools and local models.
One caveat: routed answers are advisory by default — Claude still takes the
turn unless LLM_ROUTER_ZERO_CLAUDE=1. By default some tool calls wait until a
prompt is routed; LLM_ROUTER_ENFORCE=off turns that off. Draft-rate and
acceptance figures live in docs/MEASUREMENT.md, with
their n, window and conditions.
📑 Table of Contents
Why install this
You are on a Claude Pro or Max plan, have not spent a cent beyond it, and still hit the five-hour limit — not because you asked too much, but because every prompt went to the premium model: "what does this error mean", "reformat this JSON", "is the service up" drew the same quota as the question that needed it.
Why a proxy cannot do this
Most routers here are a proxy: your agent forwards requests using your API keys. That has a hard limit — a proxy cannot intercept a session authenticated by a subscription, because there is no key to forward.
| Pays per token | Pays a subscription | |
|---|---|---|
| What runs out | your invoice | your five-hour window |
| Needs API keys | yes | no |
| A proxy can help | yes | no — nothing to intercept |
| llm-router helps | yes | yes |
Honesty as a feature
llm-router status reports verified and unverified savings
separately, each with its own n — not a blended percentage. This project does
not publish a general savings percentage; any number depends on your own
workload. See Savings below.
Quick Start
pip install llm-routing # installs the `llm-router` command
llm-router install # wire up Claude Code (default host)
llm-router doctor # check provider connectivity and setup
llm-router status # verify it's routing — est. savings vs baseline
Works with no API keys on a Claude Pro/Max subscription — routing goes through MCP tools and local models. Adding provider keys widens the pool; nothing requires them. See guide/PROVIDERS.md. To install into another host, see Works With.
Works With
| Host | Install |
|---|---|
| Claude Code (default) | llm-router install |
| Codex CLI | llm-router install --host codex |
| OpenCode | llm-router install --host opencode |
| Gemini CLI | llm-router install --host gemini-cli |
| GitHub Copilot CLI | llm-router install --host copilot-cli |
| OpenClaw | llm-router install --host openclaw |
| Trae IDE | llm-router install --host trae |
| Pi (pi.dev) | llm-router install --host pi |
| Factory Droid | llm-router install --host factory |
| Claude Desktop | llm-router install --host desktop |
| VS Code (native MCP) | llm-router install --host vscode |
| Cursor | llm-router install --host cursor |
| GitHub Copilot in VS Code (capability extension, no cost-routing) | llm-router install --host copilot |
| Windsurf / Cascade | llm-router install --host windsurf |
| Kimi Code (Moonshot AI) | llm-router install --host kimi |
llm-router install --host all installs or prints every host config in one
pass. Full per-host detail, including what each host genuinely cannot do:
guide/HOST_SUPPORT_MATRIX.md.
How It Works
Hooks intercept the prompt before your coding tool's own model sees it. A free
regex heuristic classifies it instantly, scoring about half of real prompts
with confidence — 788 of 1,571 measured (scripts/measure_low_signal_rate.py,
run 2026-09-23); the rest fall back to a default. A routable prompt gets a
draft from a free/local model first, then walks up the chain toward paid
models only if needed.
The draft is advisory — handed to Claude as an unverified hint, not a turn
replacement, unless LLM_ROUTER_ZERO_CLAUDE=1 is set. Enforcement acts at the
tool-call level: smart (default) holds selected tool calls until the prompt
is routed; off disables that. Full modes and per-host overrides:
guide/GETTING_STARTED.md.
Features
- Secrets never leave your machine. A prompt containing an API key, token or private key routes to local models only — fail-closed.
- Automatic fallback with circuit breakers. A provider that fails or rate-limits is skipped, not retried into the ground.
- You can see it working. A status line, terminal title and OS notification show the last model routed, savings and health.
- Session-end summary. Savings vs baseline, tier mix, per-provider cost, latency p50/p95/p99 and top routes.
- Media and pipelines too.
llm_image/llm_video/llm_audio, andllm_orchestratefor multi-step research.
Reference
The default MCP surface shows 12 front-door tools; LLM_ROUTER_SLIM=off
shows everything registered.
| Topic | What | Guide |
|---|---|---|
| CLI | install, status, gain, doctor, okf index/status, sessions status and more |
guide/GETTING_STARTED.md |
| Providers | Free-first — Ollama (local), OpenRouter, Gemini, Groq, your Claude subscription | guide/PROVIDERS.md |
| Routing Policies | conservative → balanced (default) → cost_aggressive; thresholds and YAML schema |
guide/POLICIES.md |
| MCP Tools | Every tool with its signature | guide/TOOLS.md |
llm-router gain today # savings so far today
llm-router okf status # knowledge base freshness
llm-router sessions status # per-session summary
Savings
Savings figures are a counterfactual — what the same tokens would have cost
at API list price, against what was actually spent — not money saved on a flat
subscription. llm-router status and llm-router savings-report split
verified from unverified savings, each with its own n. Methodology,
assumptions and limitations: docs/MEASUREMENT.md.
Trust and Security
llm-router runs entirely on your machine — no hosted proxy, nothing sent to an
llm-router service, no account required. A grounding check
(src/llm_router/grounding.py) discards any draft citing a file or function
absent from its context and the indexed repo, falling through to Claude.
LLM_ROUTER_DIRECT_EXECUTION is default on — a local model gets write_file,
edit_file and run_command under a confined, allowlisted path, and none of
that stops targeted deletes, git push --force, or exfiltration. Turn it off
with LLM_ROUTER_DIRECT_EXECUTION=false. Full analysis:
SECURITY.md.
Self-audits are published in audit/ and docs/repo_goals/AUDIT-2026-09-25.md (13 met, 6 partial, 1 not met), including negative results.
Ground Truth accumulation (opt-in, LLM_ROUTER_GROUND_TRUTH=1) builds a
corpus of replayable routing decisions instead of unlinked telemetry; off by
default since it's the only part that writes prompt text to disk. Details:
guide/GROUND_TRUTH.md.
Documentation
Full index: guide/README.md
| Document | Purpose |
|---|---|
| Quick Start (2 min) | Fastest path to working routing |
| Getting Started | Full setup walkthrough |
| Host Support Matrix | Per-host feature comparison |
| Providers | Provider setup and model recommendations |
| Routing Policies | routing.yaml schema and authoring your own policy |
| Tool Reference | All MCP tools with examples |
| Architecture | Internal design and module structure |
| Troubleshooting | Common issues and fixes |
| Testing the Router | Isolation suite for verifying routing health |
| Measurement | What "savings" means and how it's computed |
| RouterArena | Benchmark methodology, results and negative findings |
| Ground Truth | Replayable routing corpus, opt-in |
| Benchmarks | Model cost/latency/quality table, regenerated by CI |
| Changelog | Release notes (archive) |
llm-router is scored on the RouterArena
benchmark — full split, 8,400 queries, graded locally with this repo's harness,
not yet independently verified. Current Arena Score: 72.35. Full
methodology and negative results (including a proxy split that misled tuning
by 4.25 points): docs/ROUTERARENA.md.
Enterprise, Contributing, License
llm-router is built for individual developers and small teams: local cost
savings, zero ops overhead, no hosted anything. For team-wide policy
enforcement, audit export, SSO or per-org budgets, see
Chuzom — a separate, sibling product.
Contributions welcome — see CONTRIBUTING.md.
git clone https://github.com/ypollak2/llm-router.git
cd llm-router
uv sync --extra dev
uv run pytest tests/ -q # Run tests (10,000+)
uv run ruff check src/ tests/ # Lint
MIT License. Issues · Discussions · PyPI · Changelog
1 maintainer
Metadata
1 maintainer
Release files for llm-routing 15.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| llm_routing-15.4.0.tar.gz | 3.9 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| llm_routing-15.4.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 6.9 MB
Release files / llm_routing-15.4.0.tar.gz
| Download URL | llm_routing-15.4.0.tar.gz |
|---|---|
| Size | 3.9 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4686bc8f638793514c40d87635babb8a0879629e4c642ef30e34c05f10b3e6ad
|
|
BLAKE2b-256 checksum How to use checksums |
f2471d8cfb113168489d4f2ea5d9510a532ad874dbb29d0a6d740e0ad1f70f94
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 29, 2026.
Transparency logRelease files / llm_routing-15.4.0-py3-none-any.whl
| Download URL | llm_routing-15.4.0-py3-none-any.whl |
|---|---|
| Size | 3.0 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
72938a98e11bb75311b741afc6a1337f38ef712b891dba319160be1712cf6a41
|
|
BLAKE2b-256 checksum How to use checksums |
e8f9523bf6866a299c4a2f205133967e1ab641544c376b69da4986851d2d3d9a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 29, 2026.
Transparency log
