llmao-py 0.0.1

Lightweight LLM API Orchestrator - A fast, multi-provider LLM client with rate limiting and key rotation

LLMAO

Lightweight LLM API Orchestrator

One interface for all your LLM APIs, fast and simple

---

A unified Python interface for multiple LLM providers with automatic key rotation and rate limit handling. Built with Rust core for performance.

Installation

pip install llmao-py

Quick Start

from llmao import LLMClient

client = LLMClient()

response = client.completion(
    model="groq/llama-3.1-70b-versatile",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response["choices"][0]["message"]["content"])

Model Format

Use provider/model routing:

# OpenAI
client.completion(model="openai/gpt-4o", messages=[...])

# Anthropic
client.completion(model="anthropic/claude-3-5-sonnet-20241022", messages=[...])

# Groq
client.completion(model="groq/llama-3.3-70b-versatile", messages=[...])

# Cerebras
client.completion(model="cerebras/llama3.1-70b", messages=[...])

Supported Providers

View all providers

Provider	Environment Variable
OpenAI	OPENAI_API_KEY
Anthropic	ANTHROPIC_API_KEY
Groq	GROQ_API_KEY
Cerebras	CEREBRAS_API_KEY
Together	TOGETHER_API_KEY
OpenRouter	OPENROUTER_API_KEY
DeepSeek	DEEPSEEK_API_KEY
Mistral	MISTRAL_API_KEY
Fireworks	FIREWORKS_API_KEY
Perplexity	PERPLEXITY_API_KEY
SambaNova	SAMBANOVA_API_KEY
NVIDIA	NVIDIA_API_KEY
Hyperbolic	HYPERBOLIC_API_KEY
DeepInfra	DEEPINFRA_API_KEY
Novita	NOVITA_API_KEY
Xiaomi MiMo	XIAOMI_MIMO_API_KEY
Venice AI	VENICE_AI_API_KEY
GLHF	GLHF_API_KEY
Lepton	LEPTON_API_KEY
Anyscale	ANYSCALE_API_KEY
Ollama	OLLAMA_API_KEY
LM Studio	LMSTUDIO_API_KEY

Key Rotation

Automatic failover when rate limited. Configure multiple keys in your config.json:

{
  "openai/gpt-4": {
    "keys": ["sk-key1", "sk-key2", "sk-key3"],
    "rotation_strategy": "round_robin"
  }
}

Or use environment variables (suffix with _2, _3, etc.):

export OPENAI_API_KEY="sk-key1"
export OPENAI_API_KEY_2="sk-key2"
export OPENAI_API_KEY_3="sk-key3"

Configuration

LLMAO supports multiple configuration methods. See the examples/ directory for complete, runnable code for each scenario.

Contributing Providers

Want to add a new provider to LLMAO's built-in registry?

The registry.json file contains provider metadata (base URLs, default headers, etc.). Fork the repository, add your provider, and submit a pull request:

{
  "your_provider": {
    "base_url": "https://api.yourprovider.com/v1",
    "api_key_env": "YOUR_PROVIDER_API_KEY"
  }
}

Once merged, all users can use your provider without specifying base_url!

API Reference

from llmao import LLMClient, completion

# Client-based
client = LLMClient(config_path="./config.json")
client.completion(model, messages, temperature=0.7, max_tokens=100)
client.providers()  # List available providers
client.provider_info("openai")  # Get provider details

# Quick function
completion(model, messages, **kwargs)

Development

# Build from source
pip install maturin
maturin develop

# Run tests
cargo test

License

MIT