Unified LLM Provider

A Python library providing a single, unified async interface for multiple LLM providers — OpenAI, Ollama (local), and OpenRouter — so you can switch providers without changing your application code.

Features

• Unified API — one LLMClient, three providers, identical method signatures
• Event-based streaming — streams are emitted as typed dataclass events (MessageStartEvent, TextDeltaEvent, etc.), not raw chunks
• Reasoning / "thinking" support — reasoning models get dedicated start/delta/end events
• Multimodal (vision) — messages can contain text + images; local image files are auto base64-encoded
• Per-request overrides — ChatOptions lets you override model, temperature, max_tokens, and more at call time
• Validated configuration — LLMClientConfig is a pydantic model with field/model validators

Requirements

• Python 3.12+

Installation

pip install -r requirements.txt

Quick Start

Basic chat (non-streaming)

import asyncio
from llm_client import LLMClient
from utils import LLMClientConfig

client = LLMClient(LLMClientConfig(
    provider_id="openai",
    api_key="sk-...",
    model_name="gpt-4o-mini",
))

result = asyncio.run(client.get_response("Explain async Python in one paragraph."))
print(result)

Streaming with events

import asyncio
from llm_client import LLMClient
from utils import LLMClientConfig, ChatOptions
from Events.events import (
    MessageStartEvent, TextDeltaEvent, MessageEndEvent, StreamEndEvent,
)

async def main():
    client = LLMClient(LLMClientConfig(
        provider_id="ollama",
        model_name="ornith-1.5:9b",
    ))

    async for event in client.chat_stream("Hello!"):
        if isinstance(event, MessageStartEvent):
            print("\n[stream started]")
        elif isinstance(event, TextDeltaEvent):
            print(event.content, end="")
        elif isinstance(event, MessageEndEvent):
            print(f"\n[done: {event.reason}]")

asyncio.run(main())

Passing the model per-request (optional init model)

The model is optional at init — you can provide it per-request via ChatOptions:

from utils import ChatOptions

client = LLMClient(LLMClientConfig(
    provider_id="openai",
    api_key="sk-...",
))

result = await client.chat(
    "Write a haiku",
    options=ChatOptions(model="gpt-4o-mini", temperature=0.2),
)

Vision (image) prompt

from utils import Message

messages = [Message(
    role="user",
    content=[
        {"type": "text", "text": "What is this image about?"},
        {"type": "image_url", "image_url": {"url": "photo.png"}},
    ]
)]
# Local file paths are auto base64-encoded.
# http(s) URLs and data: URLs pass through as-is.

List available models

models = await client.get_model()

API Reference

LLMClient

prompt can be a plain str or a List[Message].

LLMClientConfig (pydantic, validated)

Provider defaults: Ollama → http://localhost:11434 (no key); OpenAI → https://api.openai.com/v1; OpenRouter → https://openrouter.ai/api/v1.

ChatOptions

Per-request overrides: model, temperature, max_tokens, top_p, frequency_penalty, presence_penalty, stop, n, seed, response_format.

Message

@dataclass
class Message:
    role: Literal['user', 'assistant', 'system', 'function']
    content: Any  # str or multimodal list

Streaming Events

Supported Providers

Running the Tests

Tests load API keys from .env:

# OpenAI (needs OPENAI_API_KEY in .env)
python Test/test_openai.py

# OpenRouter (needs OPENROUTER_API_KEY in .env)
python Test/test_openrouter.py

# Ollama (needs local `ollama serve` + a vision-capable model)
python Test/test_ollama.py

Project Structure

Unified_LLM_Provider/
├── llm_client.py        # LLMClient — main facade / public entry point
├── base_client.py       # BaseClient — abstract base with shared chat & streaming logic
├── utils.py             # LLMClientConfig, ChatOptions, Message, ModelConfig
├── contants.py          # ProviderType enum, DefaultModelSettings
├── Provider/
│   ├── openai.py        # OpenAIClient
│   ├── ollama.py        # OllamaClient
│   └── openrouter.py    # OpenRouterClient
├── Events/
│   └── events.py        # Streaming event dataclasses
├── Test/
│   ├── test_openai.py
│   ├── test_ollama.py
│   └── test_openrouter.py
├── .env                 # API keys (not committed)
├── pyproject.toml
├── requirements.txt
└── README.md