uc-llm-provider 0.5.0

Generic LLM provider abstraction — Anthropic, OpenAI, Google, Ollama and any OpenAI-compatible local model

uc-llm-provider

Generic LLM provider abstraction for Python.
Anthropic, OpenAI, Google — and any OpenAI-compatible local model (Ollama, LM Studio, vLLM, llama.cpp).

Three usage modes: Library · Server · CLI

---

Install

pip install uc-llm-provider                  # core only
pip install "uc-llm-provider[anthropic]"     # + Anthropic SDK
pip install "uc-llm-provider[openai]"        # + OpenAI SDK
pip install "uc-llm-provider[server,cli]"    # + Server + CLI
pip install "uc-llm-provider[all]"           # everything

---

Library

from uc_llm_provider import get_provider, ChatRequest, ChatMessage

# Cloud
provider = get_provider({
    "name":          "anthropic",
    "provider_type": "anthropic",
    "api_key":       "sk-ant-...",
    "default_model": "claude-sonnet-4-20250514",
})

# Local (Ollama)
provider = get_provider({
    "name":          "ollama",
    "provider_type": "ollama",
    "api_base":      "http://localhost:11434",
    "default_model": "llama3.2",
})

# Any OpenAI-compatible endpoint
provider = get_provider({
    "name":          "lm-studio",
    "provider_type": "openai_compatible",
    "api_base":      "http://localhost:1234/v1",
    "default_model": "local-model",
})

# Chat
response = await provider.chat(ChatRequest(
    messages=[ChatMessage(role="user", content="Hello")],
    max_tokens=256,
))
print(response.content)

Streaming

async for chunk in provider.chat_stream(request):
    if chunk.type == "content_delta":
        print(chunk.content, end="", flush=True)

Custom provider

from uc_llm_provider import register_provider
from uc_llm_provider.providers.template import TemplateProvider

class MyProvider(TemplateProvider):
    def _get_endpoint(self, path=""):
        return "https://api.myprovider.com/v1/chat"

register_provider("myprovider", MyProvider)

---

Server

uc-llm-server --port 12290 --provider anthropic --model claude-sonnet-4-20250514
uc-llm-server --port 12290 --provider ollama --api-base http://localhost:11434 --model llama3.2
uc-llm-server --port 12290 --config server.yaml

Endpoints:

Method	Path	Description
GET	/v1/health	Provider status
GET	/v1/capabilities	Supported tiers and features
GET	/v1/models	Model list
GET	/v1/models/{id}	Model detail
POST	/v1/chat	Chat completion
POST	/v1/chat/stream	Streaming chat (SSE)
POST	/v1/tokens/count	Token count

Config YAML:

provider: anthropic
model: claude-sonnet-4-20250514
api_key: ""          # or set UC_LLM_API_KEY
log_mode: jsonl      # jsonl | none
log_dir: ./logs
host: 0.0.0.0
cors_origins: ["*"]

---

CLI

uc-llm chat "Explain Playwright in one sentence"
uc-llm chat --interactive
uc-llm stream "Explain Playwright"
uc-llm models --provider ollama
uc-llm tokens "How many tokens is this?"
uc-llm health --provider anthropic
uc-llm config

Global options:

Option	Env	Description
--provider	UC_LLM_PROVIDER	Provider type
--model	UC_LLM_MODEL	Model
--api-key	UC_LLM_API_KEY	API key
--api-base	UC_LLM_API_BASE	API base URL
--log-mode	UC_LLM_LOG_MODE	jsonl or none

---

Provider types

provider_type	Provider
anthropic	Anthropic Claude
openai	OpenAI
google	Google Gemini
ollama	Ollama (local)
openai_compatible	Any OpenAI-compatible API
lm_studio / lmstudio	LM Studio
vllm	vLLM
llamacpp	llama.cpp server
localai	LocalAI

---

Ollama extras

from uc_llm_provider import get_provider

ollama = get_provider({"provider_type": "ollama", "api_base": "http://localhost:11434"})
models = await ollama.list_local_models()
async for status in ollama.pull_model("mistral"):
    print(status)
running = await ollama.is_running()

---

License

MIT · cuber IT service