uc-llm-provider 0.1.1

Generic LLM provider abstraction — Anthropic, OpenAI, Google, Ollama and any OpenAI-compatible local model

uc-uc-llm-provider

Generic LLM provider abstraction for Python.
Anthropic, OpenAI, Google — and any OpenAI-compatible local model (Ollama, LM Studio, vLLM, llama.cpp).

Install

pip install uc-uc-llm-provider

Quickstart

from uc_llm_provider import get_provider, ChatRequest, ChatMessage

# Cloud provider
provider = get_provider({
    "name":          "anthropic",
    "provider_type": "anthropic",
    "api_key":       "sk-ant-...",
    "default_model": "claude-sonnet-4-20250514",
})

# Local via Ollama
provider = get_provider({
    "name":          "ollama",
    "provider_type": "ollama",
    "api_base":      "http://localhost:11434",
    "default_model": "llama3.2",
})

# Any OpenAI-compatible endpoint (LM Studio, vLLM, llama.cpp …)
provider = get_provider({
    "name":          "lm-studio",
    "provider_type": "openai_compatible",
    "api_base":      "http://localhost:1234/v1",
    "default_model": "local-model",
})

response = await provider.chat(ChatRequest(
    messages=[ChatMessage(role="user", content="Hello")],
    max_tokens=256,
))
print(response.content)

Streaming

async for chunk in provider.chat_stream(request):
    if chunk.type == "content_delta":
        print(chunk.content, end="", flush=True)

Ollama extras

from uc_llm_provider import get_provider

ollama = get_provider({"provider_type": "ollama", ...})

# List installed models
models = await ollama.list_local_models()

# Pull a model
async for status in ollama.pull_model("mistral"):
    print(status)

# Check if running
running = await ollama.is_running()

Custom provider

from uc_llm_provider import register_provider
from uc_llm_provider.providers.template import TemplateProvider

class MyProvider(TemplateProvider):
    def _get_endpoint(self, path=""):
        return "https://api.myprovider.com/v1/chat"

register_provider("myprovider", MyProvider)
provider = get_provider({"provider_type": "myprovider", ...})

Supported provider_type values

Value	Provider
anthropic	Anthropic Claude
openai	OpenAI
google	Google Gemini
ollama	Ollama (local)
openai_compatible	Any OpenAI-compatible API
lm_studio / lmstudio	LM Studio
vllm	vLLM
llamacpp	llama.cpp server
localai	LocalAI

License

MIT