switchboard-router

Pick the cheapest LLM that can handle each prompt.

switchboard-router asks Switchboard, a 0.5B open router model, which tier a prompt needs: small, mid or frontier. Easy prompts go to a cheap model, and you pay frontier prices only when the prompt needs it.

pip install switchboard-router

Quickstart

You need one way to run the model. The easiest is Ollama: install it and leave it running. If the model isn't in Ollama yet, the first call pulls it (994 MB) from Hugging Face, which takes a minute or two. To do that ahead of time:

ollama pull hf.co/switchboardllm/switchboard-router-0.5b-GGUF

from switchboard_router import Router

router = Router()  # uses Ollama when it's running

router.route("Write a haiku about rain").tier
# 'small'


router.pick_model("Unscramble the following words: eglar, ngareo, kobo")
# 'anthropic/claude-opus-5.5'

From the terminal:

switchboard "Design a rate limiter for 10M requests per second across 5 regions"
{"tier": "mid", "domain": "general", "valid": true, "model": "anthropic/claude-sonnet-5.5"}

Route, then call the model

pick_model returns a model id for the tier. By default these are Claude models on Surplus, which serves every model behind one OpenAI-compatible key:

from openai import OpenAI
from switchboard_router import Router

router = Router()
llm = OpenAI(base_url="https://api.surplusllm.com/api/v1", api_key="sk-surplus-...")

prompt = "Write a Python function that reverses a string"
reply = llm.chat.completions.create(
    model=router.pick_model(prompt),
    messages=[{"role": "user", "content": prompt}],
)

Use your own models per tier (give all three, or the rest fall back to the defaults with a warning):

router = Router(models={"small": "my/small-model", "mid": "my/mid-model", "frontier": "my/frontier-model"})

Backends

Router() with no backend uses Ollama if it's running, then Transformers if it's installed, and otherwise tells you what to install.

Failing safe

If the backend can't be reached, errors out, or the model's output isn't valid JSON, the decision is frontier with valid=False and the reason in decision.error. A failed routing call costs you money, never quality. With fail_safe=False, those cases raise instead. Empty or non-string prompts always raise ValueError or TypeError.

Requests time out after 15 seconds by default (Router(timeout=5) to change it), and redirects are not followed, so an API key is never sent to another host.

Limits

• The router reads the first 2,000 characters of a prompt. A question at the very end of a long prompt may be missed.
• It reads the prompt it's given, so a prompt can try to talk it into a cheaper tier. Don't use it as a security boundary.

How good is the routing?

Switchboard v0.1 on 10,000 held-out prompts: 64.5% cheaper than sending every prompt to Claude Opus 5.5, and 5.5% of prompts were sent to a cheaper tier than they needed. Method, limits and the full results are in the model card. Try it in your browser: Switchboard demo.

License

Apache-2.0.