routiq 0.1.3

Routiq client SDK — connect to a Routiq gateway and manage it via Docker

Routiq

The LLM gateway that picks the right model, right provider, right price — every time.

Routiq is a local-first LLM gateway you run yourself. It sits between your app and LLM providers and reduces costs automatically — through caching, complexity-based routing, and cost tracking. Your prompts never leave your machine.

Supported providers: OpenAI · Anthropic · DeepSeek · Gemini

How it reduces costs:

Mechanism	What it does
Exact cache	Returns cached response for identical requests — zero provider cost
Complexity routing	Sends simple prompts to cheap models, hard ones to powerful models
Provider cache injection	Leverages native caching where providers support it

---

Quick Start

Option A — pip (Python 3.9+, no Docker needed)

# 1. Install
pip install routiq

# 2. First run — creates .env in current directory
routiq start
# → .env created. Edit it and run again.

# 3. Add your keys to .env
ROUTIQ_API_KEY=any-secret-you-choose
OPENAI_API_KEY=sk-...

# 4. Start the gateway
routiq start
# → Listening on http://localhost:8001

# 5. Health check
curl http://localhost:8001/health
# → {"status": "ok", "version": "0.1.0"}

Redis is optional — caching is disabled gracefully if Redis isn't running.
To enable caching: brew install redis && redis-server (Mac) or apt install redis (Linux).

Option B — Docker (recommended for teams, no Python needed)

# 1. Copy the env template
cp .env.example .env
# Add your keys:
# ROUTIQ_API_KEY=any-secret-you-choose
# OPENAI_API_KEY=sk-...

# 2. Start (includes Redis automatically)
docker compose up -d

# 3. Health check
curl http://localhost:8001/health
# → {"status": "ok", "version": "0.1.0"}

---

Connecting Your App

from routiq import RoutiqClient

client = RoutiqClient(
    api_key="your-ROUTIQ_API_KEY",      # the key you set in .env
    base_url="http://localhost:8001"    # optional, this is the default
)

# Let Routiq pick the model automatically based on prompt complexity
response = client.chat(messages=[{"role": "user", "content": "Hello"}])
print(response.choices[0].message.content)

# Pin a specific provider and model
client.chat(model="claude-latest", messages=[...])
client.chat(model="gemini-flash-latest", messages=[...])
client.chat(model="deepseek-latest", messages=[...])
client.chat(model="gpt-4o-latest", messages=[...])

# Async support
response = await client.achat(messages=[...])

Routiq handles provider routing internally — you never change your code when switching providers, only the model string.

Already using the OpenAI SDK? Point it directly at Routiq without installing anything extra:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8001/v1", api_key="your-ROUTIQ_API_KEY")

ROUTIQ_API_KEY is a secret you choose yourself — it never leaves your machine.
Provider keys (OPENAI_API_KEY etc.) live only in your .env and are never exposed to callers.

---

Model Aliases

Aliases are pinned — provider releases won't silently break your app.

Alias	Resolves To	Provider
gpt-4o-latest	gpt-4o-2024-11-20	OpenAI
gpt-4o-mini-latest	gpt-4o-mini	OpenAI
claude-latest	claude-sonnet-4-20250514	Anthropic
claude-haiku-latest	claude-haiku-4-5-20251001	Anthropic
deepseek-latest	deepseek-chat	DeepSeek
deepseek-reasoner-latest	deepseek-reasoner	DeepSeek
gemini-flash-latest	gemini-2.5-flash	Gemini
gemini-flash-lite-latest	gemini-2.5-flash-lite	Gemini
gemini-pro-latest	gemini-2.5-pro	Gemini
auto	complexity-based selection	—

Canonical model names (gpt-4o, gpt-4o-mini, claude-sonnet-4-20250514 etc.) also work directly.

---

Complexity Routing

When you use model="auto", Routiq classifies each prompt and routes it to the cheapest model that can handle it:

Complexity	Triggers	Default model
Simple	Short prompt, no tools	gemini-flash-latest
Medium	Moderate length or reasoning	gpt-4o-mini
Complex	Tools present, long context, keywords like debug, implement, analyze	claude-latest

Override tiers in .env:

SIMPLE_MODEL=gemini-flash-latest,gpt-4o-mini
MEDIUM_MODEL=gpt-4o-mini,claude-haiku-latest
COMPLEX_MODEL=claude-latest,gpt-4o

Models are tried left to right — if the first fails, the next is used.

---

Dashboard

http://localhost:8001/dashboard

Shows last 24 hours: total requests, cache hit rate, total spend, total saved, avg latency, breakdown by model and complexity, recent request table.

---

Environment Variables

Required:

ROUTIQ_API_KEY=any-secret-you-choose   # your gateway auth token
OPENAI_API_KEY=sk-...                  # at least one provider key

Optional:

ANTHROPIC_API_KEY=sk-ant-...
DEEPSEEK_API_KEY=...
GEMINI_API_KEY=...
REDIS_URL=redis://localhost:6379       # enables caching (fail-open if absent)
SQLITE_PATH=./routiq.db               # cost tracking database
LOG_LEVEL=INFO
SIMPLE_MODEL=gemini-flash-latest,gpt-4o-mini
MEDIUM_MODEL=gpt-4o-mini,claude-haiku-latest
COMPLEX_MODEL=claude-latest,gpt-4o

Only providers with a key set are active. Adapters for unconfigured providers are skipped at startup.

---

Feedback & Issues

→ github.com/yashbafna/routiq-feedback

---

License

MIT