Skip to main content

Universal AI Provider SDK

One consistent Python interface across eight Chinese LLM providers — with an opt-in middleware pipeline, strict capability enforcement, and zero vendor lock-in.

PyPI version Python versions License: MIT CI Ruff Checked with mypy

Installation · Quick Start · Providers · Middleware · CLI · Docs


Why UAI

Every Chinese LLM provider ships its own SDK, its own request shape, its own error taxonomy, and its own idea of what "streaming" means. Swapping providers means rewriting your integration layer.

uai-sdk gives all of them one stable surface. You build one client per provider, and every client behaves identically:

deepseek = UniversalAI(provider="deepseek")
qwen     = UniversalAI(provider="qwen")

deepseek.chat(messages=[...])    # same call signature
qwen.chat(messages=[...])        # same UnifiedResponse back

Same UnifiedRequest in, same UnifiedResponse out. Same exception hierarchy. Same middleware. The adapter layer absorbs the differences, so switching providers is a configuration change rather than a rewrite.

🔌 8 providers, 1 interface DeepSeek, Qwen, GLM, Kimi, StepFun, Doubao, MiniMax, Hunyuan
🛡️ Fail fast, not late The capability matrix rejects unsupported features before the network call — no wasted round-trips, no cryptic 400s
🧩 Opt-in middleware Retry, cache, circuit breaker, logging, metrics, tracing — composed with client.use(...), zero cost when unused
📐 Typed end to end Pydantic v2 models throughout; mypy-clean public surface
⚡ Measured, not claimed < 5 ms SDK overhead and ≥ 1,000 req/min enforced by a KPI regression suite in CI
🪶 Four dependencies pydantic, httpx, pyyaml — lazily-imported adapters keep cold start small
🧪 Offline-testable A stdlib-only MockProviderServer exercises the full pipeline without credentials
🔧 Extensible Add a provider without forking the SDK — see the PDK

📦 Installation

pip install uai-sdk
Other install methods
# Poetry
poetry add uai-sdk

# uv
uv add uai-sdk

# From source (development)
git clone https://github.com/GenAIwithMS/uai-sdk-python.git
cd uai-sdk-python
poetry install --with dev

Requirements: Python 3.9 – 3.13 · pydantic ^2.5 · httpx ^0.27 · pyyaml ^6.0


🚀 Quick Start

Set the API key for the provider you want, then go:

export DEEPSEEK_API_KEY="sk-..."
from uai import UniversalAI

client = UniversalAI(provider="deepseek", model="deepseek-chat")

response = client.chat(
    messages=[{"role": "user", "content": "Translate to English: 你好"}],
)

print(response.content)          # "Hello"
print(response.usage.total_tokens)
print(response.finish_reason)    # FinishReason.STOP

The API key is read from the provider's environment variable when api_key is omitted. Pass it explicitly if you'd rather manage credentials yourself:

client = UniversalAI(api_key="sk-...", provider="deepseek")

Credentials are scoped to one provider. An api_key you pass to the constructor is used for that client's provider and no other — it is never reused as a fallback for a different one. Build a separate client per provider, each with its own key.

Streaming

for chunk in client.chat(messages=[{"role": "user", "content": "Tell me a story"}], stream=True):
    if chunk.content:
        print(chunk.content, end="", flush=True)
    if chunk.ttft_ms:
        print(f"\n[time to first token: {chunk.ttft_ms:.0f} ms]")

Structured output

Hand it a Pydantic model; get a validated instance back on .parsed.

from pydantic import BaseModel

class Article(BaseModel):
    title: str
    tags: list[str]
    summary: str

response = client.chat(
    messages=[{"role": "user", "content": "Summarize the article: ..."}],
    output_schema=Article,
)

article = response.parsed          # -> Article
print(article.tags)

Malformed JSON raises ResponseParsingError rather than silently handing you a broken object. Streaming works too — the terminal chunk carries .parsed.

Tool calling

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

response = client.chat(
    messages=[{"role": "user", "content": "What's the weather in Beijing?"}],
    tools=tools,
)

for call in response.tool_calls or []:
    args = call.get_arguments()        # JSON string -> dict
    observation = get_weather(args["city"])

Vision

from uai.models import ChatMessage, ImageContent, ImageURL, Role

client = UniversalAI(provider="qwen", model="qwen-vl-max")

response = client.chat(messages=[
    ChatMessage(role=Role.USER, content=[
        ImageContent(image_url=ImageURL(url="https://example.com/photo.jpg")),
    ]),
    ChatMessage(role=Role.USER, content="Describe this image."),
])

Send an image to a text-only model and you get FeatureNotSupportedError before the request leaves the process.

Embeddings & rerank

qwen = UniversalAI(provider="qwen")     # reads DASHSCOPE_API_KEY

response = qwen.embed(["hello world", "你好世界"], model="text-embedding-v4")
print(response.vectors[0].dimension)
print(response.vectors[0].values[:5])

ranked = qwen.rerank(
    query="What is machine learning?",
    documents=["ML is a field of AI.", "Paris is in France."],
    model="qwen-reranker",
)
print(ranked.results[0].index)     # most relevant document

Pre-flight capability checks

client.supports("vision")                              # active provider/model
client.supports("rerank", provider="glm")              # ask about another one

🌐 Supported Providers

Provider Default model API key env var Chat Stream Tools Vision Embed Rerank Reasoning
DeepSeek deepseek-chat DEEPSEEK_API_KEY ✅ ✅ ✅ — ✅ — ✅
Qwen (DashScope) qwen-plus DASHSCOPE_API_KEY ✅ ✅ ✅ ✅ ✅ ✅ —
GLM (Zhipu) glm-4.7 BIGMODEL_API_KEY ✅ ✅ ✅ — ✅ ✅ ✅
Kimi (Moonshot) kimi-k2.5 MOONSHOT_API_KEY ✅ ✅ ✅ — — — —
StepFun stepfun-2.5 STEPFUN_API_KEY ✅ ✅ ✅ ✅ ✅ — —
Doubao (ByteDance) doubao-pro-32k DOUBAO_API_KEY ✅ ✅ ✅ ✅ ✅ — —
MiniMax minimax-m2.5 MINIMAX_API_KEY ✅ ✅ ✅ ✅ ✅ — —
Hunyuan (Tencent) hunyuan-turbo HUNYUAN_API_KEY ✅ ✅ ✅ ✅ ✅ — ✅

Capabilities are declared per model, not per provider — the table shows the union across each provider's model family. Run uai list-models <provider> for the exact per-model matrix, or see docs/providers.md.

Audio, TTS, and transcription are not implemented. Every provider reports these as False, and requesting them raises FeatureNotSupportedError. They are deliberately deferred — see the roadmap.


🧩 Middleware

Middleware is opt-in. A client with no middleware registered runs no pipeline code at all.

from uai import UniversalAI
from uai.middleware import (
    CacheMiddleware,
    CircuitBreakerMiddleware,
    LoggingMiddleware,
    MetricsMiddleware,
    RetryMiddleware,
)

client = (
    UniversalAI(provider="deepseek")
    .use(CircuitBreakerMiddleware(failure_threshold=5, reset_timeout=30.0))
    .use(RetryMiddleware(max_retries=3, base_delay=0.5))
    .use(CacheMiddleware(ttl=300))
    .use(LoggingMiddleware())
)

before_request hooks run in registration order; after_response and on_error run in reverse — so the outermost middleware sees the request first and the response last.

Middleware What it does
RetryMiddleware Exponential backoff with jitter on 429/5xx, network errors, and timeouts. Never retries auth failures. Streaming retries only before the first chunk reaches you.
CacheMiddleware In-memory TTL cache keyed on a normalized request hash. Schema-aware, so two requests differing only by output_schema never collide. Streaming bypasses it.
CircuitBreakerMiddleware Per (provider, model) closed → open → half-open state machine. Fast-fails a degraded provider instead of burning retries against it.
LoggingMiddleware Structured request/response logging. Never logs API keys or credentials.
MetricsMiddleware Prometheus-style counters, gauges, and histograms via an in-process MetricsRegistry with .render().
TracingMiddleware OpenTelemetry GenAI semantic-convention spans, recorded through a dependency-free SpanRecorder.

Writing your own is one subclass:

from uai.middleware import BaseMiddleware

class AuditLogger(BaseMiddleware):
    name = "audit"

    def before_request(self, request, context):
        audit.record(context.request_id, context.provider, context.model)
        return request

    def after_response(self, response, context):
        audit.complete(context.request_id, elapsed_ms=context.elapsed_ms)
        return response

client.use(AuditLogger())

See docs/middleware.md for the full hook contract.


⚙️ Configuration

Three layers, highest precedence first: constructor arguments → environment variables → YAML/JSON config file → built-in registry defaults.

client = UniversalAI(
    provider="qwen",
    model="qwen-plus",
    api_key="sk-...",
    timeout=60.0,       # seconds; overrides the registry and env config
    max_retries=3,      # enables retries — see below
)

max_retries is shorthand for registering a RetryMiddleware. It is composed inside everything you add with use(), so a circuit breaker still short-circuits without consuming attempts and a cache hit skips retrying entirely. Leave it unset for no retries — retrying stays opt-in. If you register a RetryMiddleware explicitly, that wins and the shorthand is dropped (nesting both would multiply your request count).

Environment variable overrides

Any registry field can be overridden without touching code:

Variable Purpose
UAI_PROVIDER_{NAME}_BASE_URL Point a provider at a proxy, gateway, or mock server
UAI_PROVIDER_{NAME}_TIMEOUT Request timeout in seconds
UAI_PROVIDER_{NAME}_MAX_RETRIES Retry ceiling
UAI_PROVIDER_{NAME}_RATE_LIMIT_RPM Requests-per-minute limit
UAI_PROVIDER_{NAME}_RATE_LIMIT_TPM Tokens-per-minute limit
UAI_PROVIDER_{NAME}_API_KEY_ENV Rename the variable holding the key
UAI_PROVIDER_{NAME}_DISABLE_{CAPABILITY} Force-disable a capability (e.g. kill-switch a broken vision endpoint)
UAI_CONFIG_PATH Path to a YAML/JSON config file
export UAI_PROVIDER_DEEPSEEK_BASE_URL="https://my-gateway.internal/v1"
export UAI_PROVIDER_QWEN_TIMEOUT="60"
export UAI_PROVIDER_GLM_DISABLE_RERANK="1"
Config file
# uai.yaml  — discovered via UAI_CONFIG_PATH or the default search paths
providers:
  deepseek:
    base_url: https://my-gateway.internal/v1
    timeout: 60.0
    max_retries: 5
  my-internal-llm:
    display_name: Internal Model Gateway
    base_url: https://llm.internal/v1
    api_key_env_var: INTERNAL_LLM_KEY
    default_model: internal-chat

User-supplied providers are deep-merged with the built-in registry — you override what you need and inherit the rest.

Full details in docs/configuration.md.


🖥️ CLI

Installing the package puts a uai command on your PATH.

uai list-providers                      # registered providers + whether a key is configured
uai list-models deepseek                # models, context windows, capabilities
uai benchmark                           # benchmark every chat model you have a key for
uai benchmark --providers deepseek,qwen --iterations 5 --json
Provider   Model                  Iter   OK  TTFT ms   Lat ms   Tok/s    Cost $  Err %
------------------------------------------------------------------------------------
deepseek   deepseek-chat             5    5      312      894    41.2  0.000418   0.0%
qwen       qwen-plus                 5    5      408     1102    33.7  0.000356   0.0%

TTFT is measured from real streamed responses; cost is derived from the registry's per-model pricing and actual token usage. See docs/benchmark.md.


🧪 Testing Without Credentials

uai.testing.MockProviderServer is a stdlib-only, in-process HTTP server that speaks the OpenAI-compatible wire protocol — so your tests exercise the real client → middleware → adapter → HTTP → parse path, offline.

import os
from uai import UniversalAI
from uai.testing import MockProviderServer

with MockProviderServer(latency_ms=5.0) as server:
    os.environ["UAI_PROVIDER_DEEPSEEK_BASE_URL"] = server.base_url
    client = UniversalAI(api_key="test", provider="deepseek")

    response = client.chat(messages=[{"role": "user", "content": "Hello"}])
    assert response.content

    server.fail_with(429, count=2)      # inject transient failures, then recover

📊 Performance

Non-functional guarantees are enforced by a KPI regression suite that runs in CI on every push.

KPI Target
SDK processing overhead per request < 5 ms
Streaming TTFT handling overhead < 30 ms
Throughput ≥ 1,000 req/min
Idle memory footprint (marginal over interpreter) < 50 MB
Memory under ~100 parallel requests < 150 MB

Provider adapters are imported lazily, so an application using one provider never pays the import cost of the other seven. Details and tuning knobs in docs/performance.md.


🚨 Error Handling

Every SDK failure derives from UAIError, so one except clause is enough — and each subclass carries provider, model, status_code, and response_body for triage.

from uai import UAIError, UAIRateLimitError, FeatureNotSupportedError

try:
    response = client.chat(messages=[{"role": "user", "content": "Hello"}])
except FeatureNotSupportedError as e:
    print(f"{e.feature} unavailable on {e.provider}/{e.model}")
except UAIRateLimitError as e:
    time.sleep(e.retry_after or 5)
except UAIError as e:
    print(f"[{e.status_code}] {e}")
UAIError
├── UAIAuthenticationError      401 / invalid credentials
├── UAIRateLimitError           429 — carries .retry_after
├── UAINetworkError             connection failures
├── UAITimeoutError             request exceeded its timeout
├── UAICircuitOpenError         circuit breaker rejected the call
├── ResponseParsingError        malformed or schema-invalid response
├── FeatureNotSupportedError    capability matrix rejection
├── ConfigError                 invalid provider configuration
└── UAIErrorGroup               aggregated failures across providers

📚 Documentation

Getting started

Core API

More capabilities

Operating it


🗺️ Roadmap

Status Item
✅ Unified client, registry, and capability enforcement
✅ Eight provider adapters — chat, streaming, tools, vision, embeddings, rerank
✅ Middleware engine — retry, cache, circuit breaker, logging, metrics, tracing
✅ Structured output with Pydantic validation
✅ Benchmark CLI and KPI regression suite
🚧 Async client (AsyncUniversalAI)
🚧 Automatic cross-provider fallback and routing
⏳ Audio, TTS, and transcription
⏳ Persistent cache backends (Redis)

Have an opinion on the order? Open an issue — every issue is triaged against the roadmap.


🤝 Contributing

Contributions are welcome. Start with the Contributing Guide, and read the Code of Conduct before participating.

Adding a provider is the highest-leverage contribution and doesn't require touching the core — the Provider Development Kit walks through the adapter contract end to end.

Issues are labeled so you always know where a request stands:

Label Meaning
roadmap-aligned On the plan — we intend to build it
needs-discussion Requires design discussion first
won't-implement Out of scope for this project's direction
help-wanted We explicitly welcome a PR
good-first-issue Beginner-friendly

Full conventions in .github/LABELS.md.


📝 Changelog

Release history lives in CHANGELOG.md, following Keep a Changelog and Semantic Versioning.


📄 License

Released under the MIT License. © 2026 Muhammad Siddiq.

Built for developers who'd rather ship than rewrite their provider integration.

Metadata

Release files for uai-sdk 0.1.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for uai-sdk 0.1.5
File Size Uploaded Provenance Actions
uai_sdk-0.1.5.tar.gz 77.8 kB With provenance Details for uai_sdk-0.1.5.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for uai-sdk 0.1.5
File Interpreter ABI Platform Actions
uai_sdk-0.1.5-py3-none-any.whl Python 3 none any Details for uai_sdk-0.1.5-py3-none-any.whl

Total release size: 184.7 kB

Release files / uai_sdk-0.1.5.tar.gz

Download URL uai_sdk-0.1.5.tar.gz
Size 77.8 kB
Tags Source
SHA-256 checksum
How to use checksums
7438ec82b595c4a0c3f30775559302bb3c19f342a6e65fc703b05b8d10a93611
BLAKE2b-256 checksum
How to use checksums
13d54ec0fe65a76ebe29e4311a07324bcf6f8dd33c21579b51046fd1794e9de2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 10, 2026.

Transparency log

Release files / uai_sdk-0.1.5-py3-none-any.whl

Download URL uai_sdk-0.1.5-py3-none-any.whl
Size 106.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
6d5432fd93565997a23a6fc4b3264b02ee961d47cf6f250ed8804e9e9c20488a
BLAKE2b-256 checksum
How to use checksums
66788f89cab663f3830705b664c4ec50e0cc9bbde3f0a55eaca22acab79dad8e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 10, 2026.

Transparency log

Security for uai-sdk 0.1.5

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for uai_sdk-0.1.5.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 10, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page