inferall-ai 0.1.0

Official Python SDK for the InferAll AI gateway — one API for every model provider.

inferall-ai

Official Python SDK for the InferAll AI gateway. One API, every model provider — OpenAI, Anthropic, Google Gemini, Replicate, and more — behind a single bearer token.

The PyPI distribution name is inferall-ai; the import name is inferall. (The shorter inferall distribution name is owned by an unrelated project on PyPI.)

Zero runtime dependencies. The client uses only the Python standard library so it ships cleanly into lambdas, edge functions, and air-gapped containers.

Install

pip install inferall-ai

Requires Python 3.10+.

Quick start

from inferall import Inferall

ai = Inferall()  # reads INFERALL_API_KEY from the environment

print(ai.text("Summarize the theory of relativity in two sentences."))

Explicit credentials

ai = Inferall(api_key="sk-inferall-...")

System prompt + custom model

ai.text(
    "Translate to French: 'Where is the library?'",
    system="You are a careful translator.",
    provider="openai",
    model="gpt-4o-mini",
    temperature=0.2,
)

Multi-turn chat

ai.chat(
    [
        {"role": "user", "content": "Hello!"},
        {"role": "assistant", "content": "Hi there — how can I help?"},
        {"role": "user", "content": "What's 2 + 2?"},
    ]
)

Vision

import base64

with open("receipt.jpg", "rb") as f:
    image_b64 = base64.b64encode(f.read()).decode()

ai.vision(image_b64, "What's the total on this receipt?")

Raw responses

When you need access to tool calls, fallback metadata, or any provider-specific field, use generate() to get the full payload:

raw = ai.generate(
    provider="anthropic",
    model="claude-3-5-sonnet-latest",
    operation="chat",
    messages=[{"role": "user", "content": "..."}],
)

Environment variables

Variable	Purpose
INFERALL_API_KEY	Bearer token. Used when api_key= isn't passed to the constructor.
INFERALL_BASE_URL	Override the gateway URL (default https://api.inferall.ai).
AI_GATEWAY_KEY	Legacy fallback for INFERALL_API_KEY (recognised for back-compat).
AI_GATEWAY_URL	Legacy fallback for INFERALL_BASE_URL.

Errors

Gateway failures raise InferallError, a subclass of RuntimeError:

from inferall import Inferall, InferallError

try:
    ai.text("...")
except InferallError as e:
    print(e.status, e.body)

License

MIT