any-llm
Communicate with any LLM provider using a single, unified interface. Switch between OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, and more without changing your code.
Quickstart
pip install 'any-llm-sdk[mistral,ollama]'
export MISTRAL_API_KEY="YOUR_KEY_HERE" # or OPENAI_API_KEY, etc
from any_llm import completion
import os
# Make sure you have the appropriate environment variable set
assert os.environ.get('MISTRAL_API_KEY')
response = completion(
model="mistral-small-latest",
provider="mistral",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
That's it! Change the provider name and add provider-specific keys to switch between LLM providers.
Coming from LiteLLM? Your API keys and environment variables carry over unchanged. Install the SDK with extras for the providers you need, then update your import and model strings:
pip install 'any-llm-sdk[openai,anthropic]' # or [all] for everything# before from litellm import completion response = completion(model="openai/gpt-4o", messages=[...]) # after from any_llm import completion response = completion(model="openai:gpt-4o", messages=[...])See Supported Providers to map your existing model strings.
That's the full migration — no proxy, no extra config.
Installation
Requirements
- Python 3.11 or newer
- API keys for whichever LLM providers you want to use
Basic Installation
Install support for specific providers:
pip install 'any-llm-sdk[openai]' # Just OpenAI
pip install 'any-llm-sdk[mistral,ollama]' # Multiple providers
pip install 'any-llm-sdk[all]' # All supported providers
See our list of supported providers to choose which ones you need.
Using an OpenAI-compatible gateway or local server that isn't listed? You don't need a dedicated provider entry: see Custom OpenAI-compatible Endpoints.
Setting Up API Keys
Set environment variables for your chosen providers:
export OPENAI_API_KEY="your-key-here"
export ANTHROPIC_API_KEY="your-key-here"
export MISTRAL_API_KEY="your-key-here"
# ... etc
Alternatively, pass API keys directly in your code (see Usage examples).
Otari Gateway
For budget management, API key management, usage analytics, and multi-tenant support, see mozilla-ai/otari.
Why choose any-llm?
- Simple, unified interface - Single function for all providers, switch models with just a string change
- Developer friendly - Full type hints for better IDE support and clear, actionable error messages
- Leverages official provider SDKs - Ensures maximum compatibility
- Stays framework-agnostic so it can be used across different projects and use cases
- Battle-tested - Powers our own production tools (any-agent)
Usage
any-llm offers two main approaches for interacting with LLM providers:
Option 1: Direct API Functions (Recommended for Bootstrapping and Experimentation)
Recommended approach: Use separate provider and model parameters:
from any_llm import completion
import os
# Make sure you have the appropriate environment variable set
assert os.environ.get('MISTRAL_API_KEY')
response = completion(
model="mistral-small-latest",
provider="mistral",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
Alternative syntax: Use combined provider:model format:
response = completion(
model="mistral:mistral-small-latest", # <provider_id>:<model_id>
messages=[{"role": "user", "content": "Hello!"}]
)
Option 2: AnyLLM Class (Recommended for Production)
For applications that need to reuse providers, perform multiple operations, or require more control:
from any_llm import AnyLLM
llm = AnyLLM.create("mistral", api_key="your-mistral-api-key")
response = llm.completion(
model="mistral-small-latest",
messages=[{"role": "user", "content": "Hello!"}]
)
When to Use Which Approach
| Approach | Best For | Connection Handling |
|---|---|---|
Direct API Functions (completion) |
Scripts, notebooks, single requests | New client per call (stateless) |
AnyLLM Class (AnyLLM.create) |
Production apps, multiple requests | Reuses client (connection pooling) |
Both approaches support identical features: streaming, tools, responses API, etc.
Responses API
For providers that implement the OpenAI-style Responses API, use responses or aresponses:
from any_llm import responses
result = responses(
model="gpt-4o-mini",
provider="openai",
input_data=[
{"role": "user", "content": [
{"type": "text", "text": "Summarize this in one sentence."}
]}
],
)
# Non-streaming returns an OpenAI-compatible Responses object alias
print(result.output_text)
Finding the Right Model
The provider_id should match our supported provider names.
The model_id is passed directly to the provider. To find available models:
- Check the provider's documentation
- Use our
list_modelsAPI (if the provider supports it)
Motivation
The landscape of LLM provider interfaces is fragmented. While OpenAI's API has become the de facto standard, providers implement slight variations in parameter names, response formats, and feature sets. This creates a need for light wrappers that gracefully handle these differences while maintaining a consistent interface.
Existing Solutions and Their Limitations:
- LiteLLM: Popular but reimplements provider interfaces rather than leveraging official SDKs, leading to potential compatibility issues.
- AISuite: Clean, modular approach but lacks active maintenance, comprehensive testing, and modern Python typing standards.
- Framework-specific solutions: Some agent frameworks either depend on LiteLLM or implement their own provider integrations, creating fragmentation
- Proxy Only Solutions: solutions like OpenRouter and Portkey require a hosted proxy between your code and the LLM provider.
any-llm addresses these challenges by leveraging official SDKs when available, maintaining framework-agnostic design, and requiring no proxy servers.
Documentation
- Full Documentation - Complete guides and API reference
- Supported Providers - List of all supported LLM providers
- Cookbook Examples - In-depth usage examples
Contributing
We welcome contributions from developers of all skill levels! Please see our Contributing Guide or open an issue to discuss changes.
License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Release files for any-llm-sdk 1.32.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| any_llm_sdk-1.32.1.tar.gz | 244.6 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| any_llm_sdk-1.32.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 547.9 kB
Release files / any_llm_sdk-1.32.1.tar.gz
| Download URL | any_llm_sdk-1.32.1.tar.gz |
|---|---|
| Size | 244.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
dc4fc660f51a2de64dfb877e2c5a24e56a34958d8712e4e6e3e87840aaf48c51
|
|
BLAKE2b-256 checksum How to use checksums |
db2594222f84b6a6ca46a57c099879ee965ee456cb8e65a367db670b4e828e06
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 8, 2026.
Transparency logRelease files / any_llm_sdk-1.32.1-py3-none-any.whl
| Download URL | any_llm_sdk-1.32.1-py3-none-any.whl |
|---|---|
| Size | 303.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
812aef5cc06d1414086d06ec3b2c8da934002a9b8e46a3f6d141395fd23f68b6
|
|
BLAKE2b-256 checksum How to use checksums |
eec01cef582b5f5151931c8e7b1e5a1c197fdb1f01da5467ca699bfd969ff279
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 8, 2026.
Transparency log
