indoxrouter 0.1.1

A unified interface for various LLM providers

indoxRouter

A unified interface for various LLM providers

Overview

indoxRouter is a powerful Python library that provides a unified interface to interact with various Large Language Model (LLM) providers. With a single API key, you can access models from OpenAI, Anthropic, Mistral, Google, and more, without having to manage multiple provider-specific API keys and implementations.

Key Features

• Single API for Multiple Providers: Access models from OpenAI, Anthropic, Mistral, Google, and more with a single API key
• Standardized Response Format: Consistent response format across all providers
• Streaming Support: Stream responses for real-time applications
• Cost Tracking: Track token usage and costs across providers
• Rate Limiting: Built-in rate limiting to prevent exceeding provider quotas
• Error Handling: Comprehensive error handling with detailed error messages
• Authentication: Secure authentication with JWT tokens
• Type Hints: Full type hints for better IDE support

Installation

pip install indoxRouter

For development:

pip install indoxRouter[dev]

Quick Start

from indoxRouter import Client
from indoxRouter.models import ChatMessage

# Initialize the client with your API key
client = Client(api_key="your-api-key")

# Chat completion with OpenAI
response = client.chat(
    messages=[
        {"role": "user", "content": "What are three fun activities to do in New York?"}
    ],
    model="openai/gpt-4o-mini",
    temperature=0.7,
    max_tokens=500,
)

print(response.data)

Available Providers

indoxRouter supports the following providers:

• OpenAI
• Anthropic (Claude)
• Mistral
• Google
• Cohere

Core Features

Chat Completion

response = client.chat(
    messages=[
        {"role": "user", "content": "What are three fun activities to do in New York?"}
    ],
    model="openai/gpt-4o-mini",
    temperature=0.7,
    max_tokens=500,
)

Text Completion

response = client.completion(
    prompt="Write a short story about a robot learning to paint.",
    model="anthropic/claude-3-haiku",
    temperature=0.7,
    max_tokens=500,
)

Embeddings

response = client.embeddings(
    text="This is a sample text to embed.",
    model="openai/text-embedding-3-small",
)

Image Generation

response = client.image(
    prompt="A futuristic city with flying cars and neon lights",
    model="openai/dall-e-3",
    size="1024x1024",
)

Streaming

generator = client.chat(
    messages=[
        {"role": "user", "content": "Write a short story about a robot learning to paint."}
    ],
    model="openai/gpt-4o-mini",
    temperature=0.7,
    max_tokens=500,
    stream=True,
    return_generator=True,
)

for chunk in generator:
    if isinstance(chunk, dict) and chunk.get("is_usage_info"):
        # This is the final usage info
        usage_info = chunk
    else:
        # This is a content chunk
        print(chunk, end="", flush=True)

Provider and Model Information

# List all available providers
providers = client.providers()

# List all available models
models = client.models()

# List models for a specific provider
openai_models = client.models(provider="openai")

# Get information about a specific model
model_info = client.model_info(provider="openai", model="gpt-4o-mini")

Configuration

indoxRouter can be configured using environment variables or a configuration file:

# Set API key via environment variable
import os
os.environ["INDOX_ROUTER_API_KEY"] = "your-api-key"

# Or provide it directly to the client
client = Client(api_key="your-api-key")

Error Handling

indoxRouter provides comprehensive error handling:

from indoxRouter import Client
from indoxRouter.exceptions import AuthenticationError, RateLimitError, ProviderError

try:
    client = Client(api_key="invalid-api-key")
except AuthenticationError as e:
    print(f"Authentication error: {e}")

try:
    response = client.chat(
        messages=[{"role": "user", "content": "Hello"}],
        model="nonexistent-model",
    )
except ProviderError as e:
    print(f"Provider error: {e}")

Documentation

For detailed documentation, visit https://docs.indoxrouter.com.

Examples

Check out the examples directory for more examples.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.