AI LLM Router
Intelligent routing of LLM API calls across multiple providers with automatic fallbacks, cost optimization, caching, health monitoring, and retry logic.
Features
- Multi-Provider Support: OpenAI, Anthropic, and extensible for other providers
- Intelligent Routing: Priority-based, cost-optimized, and round-robin strategies
- Automatic Fallbacks: Seamless failover between providers
- Cost Optimization: Track and optimize API usage costs
- Caching: Redis-based response caching for improved performance
- Health Monitoring: Real-time provider health checks
- Retry Logic: Configurable retry mechanisms with exponential backoff
- Async Support: Full async/await support for high-performance applications
- Streaming: Support for streaming responses
- CLI Interface: Command-line tool for easy integration
- Metrics: Prometheus-compatible metrics collection
Installation
pip install ai-llm-router
Quick Start
from llm_router import LLMRouter, RouterConfig
from llm_router.providers import OpenAIProvider, AnthropicProvider
# Configure providers
config = RouterConfig(
providers=[
OpenAIProvider(api_key="your-openai-key"),
AnthropicProvider(api_key="your-anthropic-key")
],
strategy="priority"
)
# Create router
router = LLMRouter(config)
# Make a request
response = await router.chat_completion(
messages=[{"role": "user", "content": "Hello, world!"}],
model="gpt-4"
)
CLI Usage
After installation, you can use the CLI:
ai-llm-router chat --provider openai --model gpt-4 --message "Hello, world!"
Documentation
For detailed documentation, examples, and API reference, visit our GitHub repository.
Author
Sherin Joseph - LinkedIn
License
MIT License - see LICENSE file for details.
Metadata
Release files for ai-llm-router 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ai_llm_router-0.1.0.tar.gz | 28.0 kB | Without provenance | Details for ai_llm_router-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ai_llm_router-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for ai_llm_router-0.1.0-py3-none-any.whl |
Total release size: 62.3 kB
Release files / ai_llm_router-0.1.0.tar.gz
| Download URL | ai_llm_router-0.1.0.tar.gz |
|---|---|
| Size | 28.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0eecdaca136b66b713b5640a17ce6ddd717430d65f01c8e9eda6f37f10febb81
|
|
BLAKE2b-256 checksum How to use checksums |
4bd0dc31fbd7a92b1b5057ec82a90b05a16442c8e5e1d14923c64837c00ec4db
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.3
|
Release files / ai_llm_router-0.1.0-py3-none-any.whl
| Download URL | ai_llm_router-0.1.0-py3-none-any.whl |
|---|---|
| Size | 34.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ad2c8256c0d6077d7d8c8b780874bd462b18283afce0b827f6c80d0d62b1c476
|
|
BLAKE2b-256 checksum How to use checksums |
e6afc4817bbd73952cab29647eaa363fb6d56ca265454c41343654645f3f453a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.3
|
Security for ai-llm-router 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.