🔀 llm-failover-router

Resilient Multi-Provider LLM Fallback Router for High-Availability AI Applications.

📌 Why llm-failover-router?

When LLM APIs experience outages, rate-limits (HTTP 429), or network timeouts, your application crashes or hangs.

llm-failover-router provides a zero-overhead, multi-provider fallback strategy: If Provider A (e.g., DeepSeek) fails or times out, it seamlessly switches to Provider B (OpenAI) or Provider C (Local Ollama) in milliseconds!

⚡ Quick Start

pip install llm-failover-router

Usage Example:

from llm_failover_router import LLMRouter, ProviderConfig

# Configure providers with priority order
router = LLMRouter(
    providers=[
        ProviderConfig(name="deepseek", api_key="sk-...", model="deepseek-chat"),
        ProviderConfig(name="openai", api_key="sk-...", model="gpt-4o-mini"),
        ProviderConfig(name="ollama", endpoint="http://localhost:11434", model="llama3"),
    ],
    timeout_seconds=5.0
)

# Executes primary provider; automatically falls back if API fails!
response = router.generate("Explain quantum computing in simple terms.")
print(f"Provider Used: {response.provider_used}")
print(f"Content: {response.content}")

👤 Author

Mohamed Shaban (محمد شعبان العتماني) — Applied AI Engineer
🌐 Website: msalatmani.org | 📧 Email: msalatmani@gmail.com