LLMAO

Lightweight LLM API Orchestrator

One interface for all your LLM APIs, fast and simple

A unified Python interface for multiple LLM providers with automatic key rotation and rate limit handling. Built with Rust core for performance.

Installation

pip install llmao-py

Quick Start

from llmao import LLMClient

client = LLMClient()

response = client.completion(
    model="groq/llama-3.1-70b-versatile",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response["choices"][0]["message"]["content"])

Model Format

Use provider/model routing:

# OpenAI
client.completion(model="openai/gpt-4o", messages=[...])

# Anthropic
client.completion(model="anthropic/claude-3-5-sonnet-20241022", messages=[...])

# Groq
client.completion(model="groq/llama-3.3-70b-versatile", messages=[...])

# Cerebras
client.completion(model="cerebras/llama3.1-70b", messages=[...])

Supported Providers

Key Rotation

Automatic failover when rate limited. Configure multiple keys in your config.json:

{
  "openai/gpt-4": {
    "keys": ["sk-key1", "sk-key2", "sk-key3"],
    "rotation_strategy": "round_robin"
  }
}

Or use environment variables (suffix with _2, _3, etc.):

export OPENAI_API_KEY="sk-key1"
export OPENAI_API_KEY_2="sk-key2"
export OPENAI_API_KEY_3="sk-key3"

Configuration

LLMAO supports multiple configuration methods. See the examples/ directory for complete, runnable code for each scenario.

Contributing Providers

Want to add a new provider to LLMAO's built-in registry?

The registry.json file contains provider metadata (base URLs, default headers, etc.). Fork the repository, add your provider, and submit a pull request:

{
  "your_provider": {
    "base_url": "https://api.yourprovider.com/v1",
    "api_key_env": "YOUR_PROVIDER_API_KEY"
  }
}

Once merged, all users can use your provider without specifying base_url!

API Reference

from llmao import LLMClient, completion

# Client-based
client = LLMClient(config_path="./config.json")
client.completion(model, messages, temperature=0.7, max_tokens=100)
client.providers()  # List available providers
client.provider_info("openai")  # Get provider details

# Quick function
completion(model, messages, **kwargs)

Development

# Build from source
pip install maturin
maturin develop

# Run tests
cargo test

License

MIT