routiq 0.1.5

Routiq client SDK — connect to a Routiq gateway and manage it via Docker

Routiq

The LLM gateway that picks the right model, right provider, right price — every time.

Routiq is a local-first LLM gateway you run yourself. It sits between your app and LLM providers and reduces costs automatically — through caching, complexity-based routing, and cost tracking. Your prompts never leave your machine.

Supported providers: OpenAI · Anthropic · DeepSeek · Gemini

How it reduces costs:

Mechanism	What it does	Typical saving
Exact cache	Returns cached response for identical requests — zero provider cost	20–40%
Semantic cache	Matches similar prompts using embeddings — catches paraphrases	20–40%
Complexity routing	Sends simple prompts to cheap models, hard ones to powerful models	40–60%

Real numbers from testing: 84% cache hit rate · up to 88% cost savings · 9ms p50 overhead

---

Quick Start

Tip: create a dedicated folder for Routiq before installing — both paths create config files in your current directory.

mkdir routiq && cd routiq

Option A — Docker (recommended, one command, includes Redis)

Prerequisites: Docker Desktop (Mac/Windows) or Docker Engine with docker compose (Linux).

# 1. Download install files
curl -O https://raw.githubusercontent.com/yashbafna/routiq-feedback/main/docker-compose.yml
curl -O https://raw.githubusercontent.com/yashbafna/routiq-feedback/main/.env.example
mv .env.example .env

# 2. Edit .env — set your gateway key and at least one provider key
#    ROUTIQ_API_KEY=any-secret-you-choose
#    GEMINI_API_KEY=AIza...   ← free at https://ai.google.dev

# 3. Start (Redis included automatically)
docker compose up -d

# 4. Health check
sleep 5 && curl http://localhost:8001/health
# → {"status": "ok", "version": "0.1.5"}

# 5. Make your first call
curl http://localhost:8001/v1/chat/completions \
  -H "Authorization: Bearer YOUR_ROUTIQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Say hi"}]}'

Option B — pip (Python 3.9+)

Caching note: the pip path runs Routiq without caching unless you install Redis separately. You'll still get routing and cost tracking — just not the full cache hit rate. Use Option A for the full experience in one command.

Recommended: use pipx or a virtual environment — pip install against system Python fails on macOS and Ubuntu 22.04+.

# 1. Install
pipx install routiq
# Or: python3 -m venv venv && source venv/bin/activate && pip install routiq

# 2. First run — creates .env in current directory
routiq start
# → .env created. Edit it and run again.

# 3. Edit .env
#    ROUTIQ_API_KEY=any-secret-you-choose
#    GEMINI_API_KEY=AIza...   ← free at https://ai.google.dev

# 4. Start the gateway
routiq start
# → Gateway healthy at http://localhost:8001

# 5. Health check
curl http://localhost:8001/health
# → {"status": "ok", "version": "0.1.5"}

# 6. Make your first call
curl http://localhost:8001/v1/chat/completions \
  -H "Authorization: Bearer YOUR_ROUTIQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Say hi"}]}'

To enable caching:

brew install redis && brew services start redis        # macOS
sudo apt install redis-server && sudo systemctl start redis  # Linux

Restart routiq start after Redis is up — it auto-detects Redis at localhost:6379.

---

Troubleshooting

Version mismatch (Docker). Your Docker image is cached. Force a fresh pull:

docker compose down && docker compose pull && docker compose up -d --force-recreate

.env not found. routiq start reads .env from your current working directory. cd into the folder where .env lives before running it.

"Gateway unreachable" in the dashboard. Hard-refresh your browser (Cmd+Shift+R / Ctrl+Shift+R). Usually a stale dashboard polling against a restarted backend.

No provider keys — all requests fail. At least one of GEMINI_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY, or DEEPSEEK_API_KEY must be set in .env.

---

Connecting Your App

from routiq import RoutiqClient

client = RoutiqClient(
    api_key="your-ROUTIQ_API_KEY",      # the key you set in .env
    base_url="http://localhost:8001"    # optional, this is the default
)

# Let Routiq pick the model automatically based on prompt complexity
response = client.chat(messages=[{"role": "user", "content": "Hello"}])
print(response.choices[0].message.content)

# Pin a specific provider and model
client.chat(model="claude-latest", messages=[...])
client.chat(model="gemini-flash-latest", messages=[...])
client.chat(model="deepseek-latest", messages=[...])
client.chat(model="gpt-4o-latest", messages=[...])

# Async support
response = await client.achat(messages=[...])

Already using the OpenAI SDK? Point it directly at Routiq:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8001/v1", api_key="your-ROUTIQ_API_KEY")

---

Model Aliases

Aliases are pinned — provider releases won't silently break your app.

Alias	Resolves To	Provider
gpt-4o-latest	gpt-4o-2024-11-20	OpenAI
gpt-4o-mini-latest	gpt-4o-mini	OpenAI
claude-latest	claude-sonnet-4-20250514	Anthropic
claude-haiku-latest	claude-haiku-4-5-20251001	Anthropic
deepseek-latest	deepseek-chat	DeepSeek
deepseek-reasoner-latest	deepseek-reasoner	DeepSeek
gemini-flash-latest	gemini-2.5-flash	Gemini
gemini-flash-lite-latest	gemini-2.5-flash-lite	Gemini
gemini-pro-latest	gemini-2.5-pro	Gemini
auto	complexity-based selection	—

Canonical model names (gpt-4o, claude-sonnet-4-20250514 etc.) also work directly.

---

Complexity Routing

When you use model="auto", Routiq classifies each prompt and routes it to the cheapest model that can handle it:

Complexity	Triggers	Default model
Simple	Short prompt, no tools	gemini-flash-latest
Medium	Moderate length or reasoning	gpt-4o-mini
Complex	Tools present, long context, keywords like debug, implement, analyze	claude-latest

Override tiers in .env:

SIMPLE_MODEL=gemini-flash-latest,gpt-4o-mini
MEDIUM_MODEL=gpt-4o-mini,claude-haiku-latest
COMPLEX_MODEL=claude-latest,gpt-4o

Models are tried left to right — if the first fails, the next is used.

---

Dashboard

Open http://localhost:8001/dashboard to see live stats: total requests, cache hit rate, total spend, total saved, latency (p50/p95/p99), breakdown by model and complexity, recent request table with MISS / HIT / SIMILAR badges.

---

Environment Variables

Required:

ROUTIQ_API_KEY=any-secret-you-choose
GEMINI_API_KEY=AIza...               # easiest — free at ai.google.dev

Optional:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
DEEPSEEK_API_KEY=...
REDIS_URL=redis://localhost:6379
SQLITE_PATH=./data/routiq.db
LOG_LEVEL=INFO
SEMANTIC_CACHE_ENABLED=true
SEMANTIC_THRESHOLD=0.82
SIMPLE_MODEL=gemini-flash-latest,gpt-4o-mini
MEDIUM_MODEL=gpt-4o-mini,claude-haiku-latest
COMPLEX_MODEL=claude-latest,gpt-4o

Only providers with a key set are active. Adapters for unconfigured providers are skipped at startup.

---

Feedback & Issues

→ github.com/yashbafna/routiq-feedback

---

License

MIT