modelgate 0.1.1

Type-safe, zero-SDK adapter layer that normalizes any LLM provider into one consistent interface.

unifai

A minimalist, model-agnostic adapter layer for LLMs. No massive SDKs, strict type-safe normalization, zero-overhead abstraction — just pydantic, httpx, and boto3. Unlike LiteLLM, unifai calls provider APIs directly rather than wrapping heavyweight SDKs, giving you a predictable canonical schema with nothing hidden underneath.

Install

pip install -e .

# With dev dependencies
pip install -e ".[dev]"

Quick Start

import asyncio
from unifai import ModelGate, ModelGateConfig

async def main():
    client = ModelGate(ModelGateConfig(
        openai_api_key="sk-...",
        anthropic_api_key="sk-ant-...",
    ))

    # Non-streaming
    response = await client.chat(
        model="anthropic/claude-3-5-sonnet-20241022",
        messages=[{"role": "user", "content": "What is 2+2?"}],
    )
    print(response.text)       # "4"
    print(response.tool_calls) # [] — same shape for ALL providers

    # Streaming (yields ContentBlock chunks, then a final Usage)
    async for chunk in client.stream(
        model="openai/gpt-4o",
        messages=[{"role": "user", "content": "Tell me a story"}],
    ):
        if chunk.type == "text":
            print(chunk.text, end="", flush=True)

asyncio.run(main())

Supported Providers

Provider	Model string format	Adapter	Status
OpenAI	openai/<model-id>	OpenAIAdapter	✅ Full
Anthropic	anthropic/<model-id>	AnthropicAdapter	✅ Full
AWS Bedrock	bedrock/<model-id>	BedrockAdapter	✅ Full
Groq	groq/<model-id>	GenericOpenAIAdapter	✅ Full
Ollama	ollama/<model-id>	GenericOpenAIAdapter	✅ Full
Gemini	gemini/<model-id>	GeminiAdapter	✅ Full
Vertex AI	vertex/<model-id>	VertexAdapter	⚠️ Not Tested

Any OpenAI-compatible API can be added by pointing GenericOpenAIAdapter at a new base_url — no new adapter code required.

Tool Use

Tools produce the exact same ContentBlock shape regardless of provider:

from unifai import Tool, ToolParameter

weather_tool = Tool(
    name="get_weather",
    description="Get current weather for a location",
    parameters={
        "location": ToolParameter(type="string", description="City name"),
    },
    required=["location"],
)

response = await client.chat(
    model="anthropic/claude-3-5-sonnet-20241022",
    messages=[{"role": "user", "content": "Weather in NYC?"}],
    tools=[weather_tool],
)

for tc in response.tool_calls:
    print(tc.tool_name)   # "get_weather"
    print(tc.tool_input)  # {"location": "NYC"} — always a dict, never a string

Error Handling

Every adapter catches raw httpx.HTTPStatusError and re-raises as a typed ModelGateError — provider-specific error formats never leak to your code:

ModelGateError
├── AuthenticationError   # 401 — invalid or missing API key
├── RateLimitError        # 429 — provider rate limit exceeded
├── InvalidRequestError   # 400 — malformed input
├── ProviderError         # 5xx — unexpected provider failure
│   ├── BedrockError
│   └── VertexError
└── StreamingError        # error mid-stream

from unifai import RateLimitError, AuthenticationError

try:
    response = await client.chat(
        model="openai/gpt-4o",
        messages=[{"role": "user", "content": "Hello"}],
    )
except RateLimitError:
    # retry with backoff
except AuthenticationError:
    # bad key

Architecture

src/unifai/
├── __init__.py          # Public API surface
├── types.py             # Pydantic v2 canonical schemas
├── errors.py            # ModelGateError hierarchy
├── client.py            # ModelGate entry point + provider routing
└── providers/
    ├── base.py          # BaseProvider ABC
    ├── openai.py        # OpenAI adapter
    ├── anthropic.py     # Anthropic adapter
    ├── bedrock.py       # AWS Bedrock Converse API
    ├── gemini.py        # Gemini (stub)
    ├── vertex.py        # Vertex AI (stub)
    └── generic_openai.py  # OpenAI-compatible fallback

Testing

pytest tests/ -v

Dependencies

• pydantic — type-safe models
• httpx — async HTTP (no provider SDKs)
• boto3 — AWS credential signing only