wiseroute 0.1.0

Intelligent LLM routing — use free providers first, pay only when you must

wxw — Intelligent LLM Routing

Route LLM requests through free and cheap providers first. Pay only when you must.

---

Why wxw?

You're paying for OpenAI when 80% of your requests could run on free Gemini or Groq.

wxw automatically routes requests through cheap and free LLM providers first, falling back to premium ones only when needed. Zero configuration to get started — it reads your existing API keys.

Average savings: 70–90% for typical workloads.

---

Quick Start

pip install wxw

import wxw

router = wxw.Router()  # auto-detects providers from env vars
result = router.complete_sync([wxw.Message(role="user", content="Hello!")])
print(f"Provider: {result.provider} | Cost: ${result.cost:.6f}")
print(result.content)

Set your API keys (any combination works):

export GEMINI_API_KEY=...     # ✅ Free tier: 1500 req/day
export GROQ_API_KEY=...       # ✅ Free tier: 14K tokens/min
export DEEPSEEK_API_KEY=...   # 💰 $0.14/1M tokens
export OPENAI_API_KEY=...     # 💰 $0.15/1M tokens
export ANTHROPIC_API_KEY=...  # 💰 $0.25/1M tokens

---

Supported Providers

Provider	Free Tier	Input / 1M tokens	Speed	Env Var
Gemini Flash	✅ 1,500 req/day	$0.00	Fast	GEMINI_API_KEY
Groq	✅ 14K tok/min	$0.059	Ultra-fast	GROQ_API_KEY
DeepSeek	—	$0.14	Fast	DEEPSEEK_API_KEY
Mistral	—	$0.20	Fast	MISTRAL_API_KEY
GPT-4o-mini	—	$0.15	Medium	OPENAI_API_KEY
Claude Haiku	—	$0.25	Medium	ANTHROPIC_API_KEY
Ollama	✅ Unlimited	$0.00	Varies	(local)

---

Features

• 🔄 Automatic fallback — tries providers in order, moves to next on any failure
• 💰 Real-time cost tracking — per request, per provider, session totals
• ⚡ Async + streaming — native async with streaming token support
• 🎯 Routing strategies — cheapest, fastest, smart (error-rate-aware), round-robin
• 🧩 Middleware — logging, caching, retry with exponential backoff, transforms
• 📊 Cost analytics — export to JSON or CSV
• 🏠 Local-first — Ollama support for completely free private inference
• 0️⃣ Minimal deps — only httpx + pydantic, no heavy provider SDKs required

---

Routing Strategies

# Default — cheapest provider first
router = wxw.Router(strategy="cheapest")

# Lowest latency — sort by priority
router = wxw.Router(strategy="fastest")

# Adaptive — skips providers with >30% error rate
router = wxw.Router(strategy="smart")

# Load balancing — distributes evenly across providers
router = wxw.Router(strategy="round_robin")

---

Cost Tracking

router = wxw.Router()

for question in ["Hello", "What is AI?", "Explain Python"]:
    router.complete_sync([wxw.Message(role="user", content=question)])

# Session summary
print(router.cost_summary)
# {
#   "total_requests": 3,
#   "total_tokens": 542,
#   "total_cost_usd": 0.0,       # ← all served by free Gemini
#   "avg_latency_ms": 318.4,
#   "providers": {"gemini": {...}}
# }

# Export
router.tracker.to_csv()   # CSV report
router.tracker.to_json()  # JSON report

---

Streaming

import asyncio
import wxw

async def main():
    router = wxw.Router()
    async for token in router.stream([wxw.Message(role="user", content="Tell me a story")]):
        print(token, end="", flush=True)

asyncio.run(main())

---

Middleware

from wxw.middleware import LoggingMiddleware, CacheMiddleware

router = wxw.Router(
    middleware=[
        LoggingMiddleware(),         # log every request
        CacheMiddleware(ttl=300),    # cache for 5 minutes
    ]
)

---

Local-First with Ollama

from wxw.models import ProviderConfig

router = wxw.Router(
    providers=[
        ProviderConfig(name="ollama", model="llama3.2", priority=0),   # free local
        ProviderConfig(name="gemini", model="gemini-2.0-flash", priority=1),  # free cloud
        ProviderConfig(name="openai", model="gpt-4o-mini", priority=2),       # paid fallback
    ],
    strategy="fastest",
)

---

Explicit Provider Chain

from wxw import Router
from wxw.models import ProviderConfig

router = Router(
    providers=[
        ProviderConfig(name="gemini",   model="gemini-2.0-flash",        priority=0),
        ProviderConfig(name="groq",     model="llama-3.3-70b-versatile", priority=1),
        ProviderConfig(name="deepseek", model="deepseek-chat",           priority=2),
        ProviderConfig(name="openai",   model="gpt-4o-mini",             priority=3),
    ]
)

---

Installation

Minimal (recommended):

pip install wxw

With all optional provider SDKs:

pip install wxw[all]

Development:

git clone https://github.com/Koski-ai/wxw.git
cd wxw
pip install -e ".[dev]"
pytest

---

Documentation

• Getting Started
• Providers
• Routing Strategies
• Cost Tracking
• Middleware

---

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

The fastest way to contribute is to add a new provider — see the existing implementations in wxw/providers/ for the pattern.

---

License

MIT — see LICENSE

---

wxw = Wise eXchange Workflow — route wisely, spend less.