Project logo

any-llm

Read the Blog Post

Docs

Linting Unit Tests Integration Tests

Python 3.11+ PyPI Discord

Communicate with any LLM provider using a single, unified interface. Switch between OpenAI, Anthropic, Azure / Microsoft Foundry, Mistral, Ollama, and more without changing your code.

Documentation | otari.ai | Try the Demos | Contributing

Quickstart

pip install 'any-llm-sdk[mistral,ollama]'

export MISTRAL_API_KEY="YOUR_KEY_HERE"  # or OPENAI_API_KEY, etc
from any_llm import completion
import os

# Make sure you have the appropriate environment variable set
assert os.environ.get('MISTRAL_API_KEY')

response = completion(
    model="mistral-small-latest",
    provider="mistral",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

That's it! Change the provider name and add provider-specific keys to switch between LLM providers.

Coming from LiteLLM? Your API keys and environment variables carry over unchanged. Install the SDK with extras for the providers you need, then update your import and model strings:

pip install 'any-llm-sdk[openai,anthropic]'  # or [all] for everything

# before
from litellm import completion
response = completion(model="openai/gpt-4o", messages=[...])

# after
from any_llm import completion
response = completion(model="openai:gpt-4o", messages=[...])

See Supported Providers to map your existing model strings.

That's the full migration — no proxy, no extra config.

Installation

Requirements

  • Python 3.11 or newer
  • API keys for whichever LLM providers you want to use

Basic Installation

Install support for specific providers:

pip install 'any-llm-sdk[openai]'           # Just OpenAI
pip install 'any-llm-sdk[mistral,ollama]'   # Multiple providers
pip install 'any-llm-sdk[all]'              # All supported providers

See our list of supported providers to choose which ones you need.

Setting Up API Keys

Set environment variables for your chosen providers:

export OPENAI_API_KEY="your-key-here"
export ANTHROPIC_API_KEY="your-key-here"
export MISTRAL_API_KEY="your-key-here"
# ... etc

Alternatively, pass API keys directly in your code (see Usage examples).

Otari Gateway

For budget management, API key management, usage analytics, and multi-tenant support, see mozilla-ai/otari.

Why choose any-llm?

  • Simple, unified interface - Single function for all providers, switch models with just a string change
  • Developer friendly - Full type hints for better IDE support and clear, actionable error messages
  • Leverages official provider SDKs - Ensures maximum compatibility
  • Stays framework-agnostic so it can be used across different projects and use cases
  • Battle-tested - Powers our own production tools (any-agent)

Usage

any-llm offers two main approaches for interacting with LLM providers:

Option 1: Direct API Functions (Recommended for Bootstrapping and Experimentation)

Recommended approach: Use separate provider and model parameters:

from any_llm import completion
import os

# Make sure you have the appropriate environment variable set
assert os.environ.get('MISTRAL_API_KEY')

response = completion(
    model="mistral-small-latest",
    provider="mistral",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Alternative syntax: Use combined provider:model format:

response = completion(
    model="mistral:mistral-small-latest", # <provider_id>:<model_id>
    messages=[{"role": "user", "content": "Hello!"}]
)

Option 2: AnyLLM Class (Recommended for Production)

For applications that need to reuse providers, perform multiple operations, or require more control:

from any_llm import AnyLLM

llm = AnyLLM.create("mistral", api_key="your-mistral-api-key")

response = llm.completion(
    model="mistral-small-latest",
    messages=[{"role": "user", "content": "Hello!"}]
)

When to Use Which Approach

Approach Best For Connection Handling
Direct API Functions (completion) Scripts, notebooks, single requests New client per call (stateless)
AnyLLM Class (AnyLLM.create) Production apps, multiple requests Reuses client (connection pooling)

Both approaches support identical features: streaming, tools, responses API, etc.

Responses API

For providers that implement the OpenAI-style Responses API, use responses or aresponses:

from any_llm import responses

result = responses(
    model="gpt-4o-mini",
    provider="openai",
    input_data=[
        {"role": "user", "content": [
            {"type": "text", "text": "Summarize this in one sentence."}
        ]}
    ],
)

# Non-streaming returns an OpenAI-compatible Responses object alias
print(result.output_text)

Finding the Right Model

The provider_id should match our supported provider names.

The model_id is passed directly to the provider. To find available models:

  • Check the provider's documentation
  • Use our list_models API (if the provider supports it)

Motivation

The landscape of LLM provider interfaces is fragmented. While OpenAI's API has become the de facto standard, providers implement slight variations in parameter names, response formats, and feature sets. This creates a need for light wrappers that gracefully handle these differences while maintaining a consistent interface.

Existing Solutions and Their Limitations:

  • LiteLLM: Popular but reimplements provider interfaces rather than leveraging official SDKs, leading to potential compatibility issues.
  • AISuite: Clean, modular approach but lacks active maintenance, comprehensive testing, and modern Python typing standards.
  • Framework-specific solutions: Some agent frameworks either depend on LiteLLM or implement their own provider integrations, creating fragmentation
  • Proxy Only Solutions: solutions like OpenRouter and Portkey require a hosted proxy between your code and the LLM provider.

any-llm addresses these challenges by leveraging official SDKs when available, maintaining framework-agnostic design, and requiring no proxy servers.

Documentation

Contributing

We welcome contributions from developers of all skill levels! Please see our Contributing Guide or open an issue to discuss changes.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Project links

Data verified by PyPI on Jul 29, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.26.0)

2 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for daavoo from gravatar.com daavoo Avatar for njbrake from gravatar.com njbrake

GitHub Statistics

Data verified by PyPI on Jul 29, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

Apache-2.0

Requires

Python >=3.11

Provides Extra

all platform perplexity mistral anthropic gemini vertexai vertexaianthropic azureanthropic huggingface cohere cerebras groq bedrock azure watsonx together ollama voyage xai sagemaker lmstudio atlascloud azureopenai cascadia databricks deepseek fireworks gmi otari github inception kenari llama llamacpp llamafile moonshot nebius neosantara openai openrouter portkey qiniu requesty sambanova minimax mzai vllm zai dashscope deepinfra edenai telnyx

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

any_llm_sdk-1.23.0.tar.gz (165.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

any_llm_sdk-1.23.0-py3-none-any.whl (218.5 kB view details)

Uploaded Python 3

File details

Details for the file any_llm_sdk-1.23.0.tar.gz.

File metadata

  • Download URL: any_llm_sdk-1.23.0.tar.gz
  • Upload date:
  • Size: 165.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for any_llm_sdk-1.23.0.tar.gz
Algorithm Hash digest
SHA256 e4291744445054d3af5ff745a12716a214bef92540fab397332b30ed791ddcc9
MD5 6fd220bef9266fc0980cc9cf68dbb155
BLAKE2b-256 24e69745d988d25019a35ee5b6294b2266900e5ba2a616b676b062178947ddb3

See more details on using hashes here.

Provenance

The following attestation bundles were made for any_llm_sdk-1.23.0.tar.gz:

Publisher: release.yaml on mozilla-ai/any-llm

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file any_llm_sdk-1.23.0-py3-none-any.whl.

File metadata

  • Download URL: any_llm_sdk-1.23.0-py3-none-any.whl
  • Upload date:
  • Size: 218.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for any_llm_sdk-1.23.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ddb225ae17497c03369243113184d7f2ed4757cb9a8b55517168dfb96d5ec3b1
MD5 5cc9d3b3ef87dbd94c72fa8ef42a2dae
BLAKE2b-256 985d6633d59f00322fe440ef2aba4ed2ae64ea5221f680fc86d24f39cf7cc96b

See more details on using hashes here.

Provenance

The following attestation bundles were made for any_llm_sdk-1.23.0-py3-none-any.whl:

Publisher: release.yaml on mozilla-ai/any-llm

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

1.26.0

2 files

1.25.0

2 files

1.24.0

2 files

This release

1.23.0 This release

2 files

1.22.1

2 files

1.22.0

2 files

1.21.0

2 files

1.20.0

2 files

1.19.0

2 files

1.18.0

2 files

1.17.0

2 files

1.16.0

2 files

1.15.0

2 files

1.14.0

2 files

1.13.0

2 files

1.12.1

2 files

1.11.1

2 files

1.11.0

2 files

1.10.0

2 files

1.9.0

2 files

1.8.6

2 files

1.8.5

2 files

1.8.4

2 files

1.8.3

2 files

1.8.2

2 files

1.8.1

2 files

1.8.0

2 files

1.7.0

2 files

1.6.2

2 files

1.6.1

2 files

1.6.0

2 files

1.5.0

2 files

1.4.3

2 files

1.4.2

2 files

1.4.1

2 files

1.4.0

2 files

1.3.0

2 files

1.2.0

2 files

1.1.0

2 files

1.0.0

2 files

0.21.0

2 files

0.20.3

2 files

0.20.2

2 files

0.20.1

2 files

0.20.0

2 files

0.19.1

2 files

0.19.0

2 files

0.18.0

2 files

0.17.2

2 files

0.17.1

2 files

0.17.0

2 files

0.16.0

2 files

0.15.0

2 files

0.14.2

2 files

0.14.1

2 files

0.14.0

2 files

0.13.1

2 files

0.13.0

2 files

0.12.1

2 files

0.12.0

2 files

0.11.2

2 files

0.11.1

2 files

0.11.0

2 files

0.10.0

2 files

0.9.0

2 files

0.8.1

2 files

0.8.0

2 files

0.7.0

2 files

0.6.0

2 files

0.5.0

2 files

0.4.0

2 files

0.3.0

2 files

0.2.0

2 files

0.1.1

2 files

0.1.0

2 files

0.0.5

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page