univllm

A universal Python package that provides a standardised interface for different LLM providers including OpenAI, Anthropic, Deepseek, Mistral, and Gemini.

Features

• Universal Interface: Single API to interact with multiple LLM providers
• Auto-Detection: Automatically detect the appropriate provider based on model name
• Streaming Support: Stream completions from all supported providers
• MCP Tool Calling: Compatible with Model Context Protocol (MCP) for function/tool calling
• Model Capabilities: Query model capabilities like context window, function calling support, etc.
• Error Handling: Comprehensive error handling with provider-specific exceptions
• Async Support: Fully asynchronous API for better performance

Supported Providers

• OpenAI: GPT-4o, GPT-5.x & GPT-5.2 family models
• Anthropic: Claude 3.x, 4.x & 4.5 family models
• Deepseek: DeepSeek V3.2, Chat, Reasoner, Coder & VL models
• Mistral: Mistral Large 3, Ministral 3, Magistral, Codestral & specialized models
• Gemini: Google Gemini 1.5, 2.0 & 2.5 family models

Supported Model Prefixes

Provider auto-detection uses simple prefix matching (see SUPPORTED_MODELS lists). Known prefixes are used as hints for provider selection and capability metadata, but request models accept arbitrary model strings so new provider releases are not rejected before an API call. Provider-specific suffixes or date/version tags (e.g. -20240229, -latest, -0125, minor patch tags) are allowed.

Note: If a model name is not yet in the prefix lists, you can still pass it directly when you also know the provider. Adding new prefixes mainly improves auto-detection and capability lookup.

Installation

pip install univllm

Quick Start

import asyncio
from univllm import UniversalLLMClient


async def main():
    client = UniversalLLMClient()

    # Auto-detects provider based on model name
    response = await client.complete(
        messages=[{"role": "user", "content": "What is the capital of France?"}],
        model="gpt-5.2"
    )

    print(response.content)


asyncio.run(main())

Configuration

Set your API keys as environment variables:

export OPENAI_API_KEY="your-openai-key"
export ANTHROPIC_API_KEY="your-anthropic-key"
export DEEPSEEK_API_KEY="your-deepseek-key"
export MISTRAL_API_KEY="your-mistral-key"
export GEMINI_API_KEY="your-gemini-key"

Or pass them directly:

from univllm import UniversalLLMClient, ProviderType

client = UniversalLLMClient(
    provider=ProviderType.OPENAI,
    api_key="your-api-key"
)

Usage Examples

Basic Completion

import asyncio
from univllm import UniversalLLMClient


async def main():
    client = UniversalLLMClient()

    response = await client.complete(
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Explain quantum computing briefly."}
        ],
        model="gpt-5.2",
        max_tokens=150,
        temperature=0.7
    )

    print(f"Response: {response.content}")
    print(f"Provider: {response.provider}")
    print(f"Model: {response.model}")
    print(f"Usage: {response.usage}")


asyncio.run(main())

Streaming Completion

import asyncio
from univllm import UniversalLLMClient


async def main():
    client = UniversalLLMClient()

    async for chunk in client.stream_complete(
            messages=[{"role": "user", "content": "Tell me a short story about a robot."}],
            model="claude-sonnet-4-5",
            max_tokens=200
    ):
        print(chunk, end="", flush=True)


asyncio.run(main())

Model Capabilities

import asyncio
from univllm import UniversalLLMClient


async def main():
    client = UniversalLLMClient()

    # Get capabilities for a specific model
    capabilities = client.get_model_capabilities("gpt-5.2")

    print(f"Supports function calling: {capabilities.supports_function_calling}")
    print(f"Context window: {capabilities.context_window}")
    print(f"Max tokens: {capabilities.max_tokens}")

    # Get all supported models
    all_models = client.get_supported_models()
    for provider, models in all_models.items():
        print(f"{provider}: {len(models)} models")


asyncio.run(main())

Tool Calling (MCP Compatible)

univllm supports function/tool calling following the Model Context Protocol (MCP) format:

import asyncio
from univllm import UniversalLLMClient, ToolDefinition


async def main():
    client = UniversalLLMClient()

    # Define a tool using MCP format
    weather_tool = ToolDefinition(
        name="get_weather",
        description="Get current weather for a location",
        input_schema={
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City name or zip code"
                }
            },
            "required": ["location"]
        }
    )

    # Request with tools
    response = await client.complete(
        messages=[{"role": "user", "content": "What's the weather in Paris?"}],
        model="gpt-4o",
        tools=[weather_tool],
        tool_choice="auto"  # Let the model decide when to use tools
    )

    # Check if model wants to call a tool
    if response.tool_calls:
        tool_call = response.tool_calls[0]
        print(f"Tool: {tool_call.name}")
        print(f"Arguments: {tool_call.arguments}")
        
        # Execute the tool and get result
        # ... your tool execution logic ...
        
        # Continue conversation with tool result
        # ... send tool result back to model ...


asyncio.run(main())

You can also pass tools as dictionaries:

tools = [
    {
        "name": "calculate",
        "description": "Perform arithmetic calculations",
        "input_schema": {
            "type": "object",
            "properties": {
                "expression": {"type": "string"}
            },
            "required": ["expression"]
        }
    }
]

response = await client.complete(
    messages=[{"role": "user", "content": "Calculate 15 * 23"}],
    model="gpt-4o",
    tools=tools
)

See examples_tool_calling.py for more comprehensive examples.

Multiple Providers

import asyncio
from univllm import UniversalLLMClient
from univllm.models import ProviderType


async def main():
    client = UniversalLLMClient()

    question = "What is machine learning?"

    # Use separate clients or call set_provider() when switching providers.
    openai_client = UniversalLLMClient()
    anthropic_client = UniversalLLMClient()

    openai_response = await openai_client.complete(
        messages=[{"role": "user", "content": question}],
        model="gpt-5.2"
    )

    anthropic_response = await anthropic_client.complete(
        messages=[{"role": "user", "content": question}],
        model="claude-sonnet-4-5"
    )

    print(f"OpenAI: {openai_response.content[:100]}...")
    print(f"Anthropic: {anthropic_response.content[:100]}...")


asyncio.run(main())

API Reference

UniversalLLMClient

Main client class for interacting with LLM providers.

Methods

• complete(): Generate a completion
• stream_complete(): Generate a streaming completion
• get_model_capabilities(): Get model capabilities
• get_supported_models(): Get supported models for all providers
• set_provider(): Set or change the provider

Models

• CompletionRequest: Request object for completions
• CompletionResponse: Response object from completions
• ModelCapabilities: Information about model capabilities
• Message: Individual message in a conversation

Providers

• ProviderType: Enum of supported providers
• BaseLLMProvider: Base class for provider implementations

Exceptions

• UniversalLLMError: Base exception
• ProviderError: Provider-related errors
• ModelNotSupportedError: Unsupported model errors
• AuthenticationError: Authentication failures
• ConfigurationError: Configuration issues

Development

git clone https://github.com/nihilok/univllm.git
cd univllm
pip install -e ".[dev]"

Run tests:

pytest

Licence

MIT Licence