inferall-ai

Official Python SDK for the InferAll AI gateway. One API, every model provider — OpenAI, Anthropic, Google Gemini, Replicate, and more — behind a single bearer token.

The PyPI distribution name is inferall-ai; the import name is inferall. (The shorter inferall distribution name is owned by an unrelated project on PyPI.)

Zero runtime dependencies. The client uses only the Python standard library so it ships cleanly into lambdas, edge functions, and air-gapped containers.

Install

pip install inferall-ai

Requires Python 3.10+.

Quick start

from inferall import Inferall

ai = Inferall()  # reads INFERALL_API_KEY from the environment

# The built-in default is Gemini, which is a PAID model. On the no-card trial
# pass a free NVIDIA model so this first call succeeds.
print(
    ai.text(
        "Summarize the theory of relativity in two sentences.",
        provider="nvidia",
        model="nvidia/nemotron-3-ultra-550b-a55b",
    )
)

Explicit credentials

ai = Inferall(api_key="ifu_...")

System prompt + custom model

ai.text(
    "Translate to French: 'Where is the library?'",
    system="You are a careful translator.",
    provider="openai",
    model="gpt-4o-mini",
    temperature=0.2,
)

Multi-turn chat

ai.chat(
    [
        {"role": "user", "content": "Hello!"},
        {"role": "assistant", "content": "Hi there — how can I help?"},
        {"role": "user", "content": "What's 2 + 2?"},
    ]
)

Vision

import base64

with open("receipt.jpg", "rb") as f:
    image_b64 = base64.b64encode(f.read()).decode()

ai.vision(image_b64, "What's the total on this receipt?")

Raw responses

When you need access to tool calls, fallback metadata, or any provider-specific field, use generate() to get the full payload:

raw = ai.generate(
    provider="anthropic",
    model="claude-sonnet-4-6",
    operation="chat",
    messages=[{"role": "user", "content": "..."}],
)

Environment variables

Errors

Gateway failures raise InferallError, a subclass of RuntimeError:

from inferall import Inferall, InferallError

try:
    ai.text("...")
except InferallError as e:
    print(e.status, e.body)

License

MIT