infery

Official Python SDK for the infery.ai inference gateway — chat, embeddings, images, video, music, audio, 3D, files and workflows, all typed, with the deferred-job and retry behaviour handled for you.

One runtime dependency (httpx). Python 3.10+. A synchronous client and an asynchronous twin, namespace for namespace.

pip install infery

import asyncio
import os

from infery import AsyncInfery, Infery

client = Infery(api_key=os.environ["INFERY_API_KEY"])

chat = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Say hello in one line."}],
)
print(chat.choices[0]["message"]["content"])


# The same call on the async client. Same names, same arguments, same defaults.
async def main() -> None:
    async with AsyncInfery(api_key=os.environ["INFERY_API_KEY"]) as aclient:
        chat = await aclient.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": "Say hello in one line."}],
        )
        print(chat.choices[0]["message"]["content"])


asyncio.run(main())

Infery and AsyncInfery are twins by construction — the same 42 methods, the same parameters, the same defaults — and tests/test_surface_parity.py fails if one grows a method the other does not. So every example below is an await away from its async form, and only the places where the two genuinely differ are called out.

Get a key at app.infery.ai. Full documentation: docs.infery.ai, with every signature and every type on the reference.

Chat

Any model slug from GET /v1/models — OpenAI, Anthropic, Google, xAI, open weights — through one shape:

answer = client.chat.completions.create(
    model="claude-sonnet-4-5",
    messages=[
        {"role": "system", "content": "Answer in one sentence."},
        {"role": "user", "content": "Why is the sky blue?"},
    ],
    temperature=0.2,
    max_tokens=200,
)

print(answer.choices[0]["message"]["content"])
print("cost in credits:", answer.credits_used)

Tool calling, JSON mode, vision and PDF attachments work as they do on the OpenAI API. tool_calls may carry several entries — group argument fragments by index, never by list position.

Every result is a frozen dataclass with the untouched response body on raw, so a field the wire adds tomorrow is reachable today, just not by name.

Streaming

for chunk in client.chat.completions.stream(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a haiku about the sea."}],
):
    if chunk["choices"]:
        print(chunk["choices"][0]["delta"].get("content", ""), end="")

stream() is a generator: nothing is requested until you start iterating, so an unconsumed one spends nothing. On AsyncInfery it is an async generator — async for, never await.

The last chunk before the stream ends carries credits_used and an empty choices list. It is yielded like any other chunk rather than hidden, because it is the only place the cost of a streamed call appears — which is also why the guard above is if chunk["choices"] rather than an index.

A stream cut before its [DONE] terminator raises StreamTruncatedError after delivering the chunks that did arrive. Catch it if partial output is useful, but knowingly: the block most often cut is the trailing usage chunk.

Embeddings

embedding = client.embeddings.create(
    model="text-embedding-3-small",
    input=["the quick brown fox", "jumps over the lazy dog"],
)
print(len(embedding.data))

One call for every media modality

Images, video, music, audio and 3D each have their own section below, and their own method. When the modality is a runtime value though — a model picked from the catalogue, a choice in a UI, a row in a queue — a method per modality forces an if chain at every call site, and every one of them has to be edited when a modality is added.

media.generate() takes the modality as data instead:

# `modality` is exactly what GET /v1/models reports as `_infery.modality`, so it
# can come straight off a model you looked up.
result = client.media.generate(
    modality="video",  # "image" | "video" | "audio" | "music" | "object_3d" | "upscale"
    model="veo-3.1",
    prompt="a drone shot over a coastline",
    duration=8,  # model-specific params pass straight through
)

print(result.artifacts[0].url)

Switch the modality and nothing else changes:

def make(job: dict[str, str]) -> list[str | bytes | None]:
    result = client.media.generate(**job)
    return [a.url or a.b64 or a.data for a in result.artifacts]

It waits by default — that is what lets one code path serve every modality, since images answer in seconds and video takes minutes.

on_progress reports for video only. It is accepted for all six modalities and fires for one, because video is the only modality whose generation this method polls itself; the other five poll inside their per-modality resource, none of which takes a callback. media.wait(on_progress=...) reports for all six. A keyword that is accepted and silently does nothing for five of six modalities is a trap, so it is written down here rather than left to be discovered from silence:

# Reports. `video` polls here.
client.media.generate(
    modality="video",
    model="veo-3.1",
    prompt="a reef",
    on_progress=lambda p: print(p.status, p.progress),
)

# Accepted, never called. Use media.wait() if you need progress for these.
client.media.generate(
    modality="image",
    model="flux-pro",
    prompt="a reef",
    on_progress=lambda p: print(p.status, p.progress),
)

For a handle instead of a result — the same deferral Deferred jobs covers, reached through one keyword:

started = client.media.generate(modality="image", model="flux-pro", prompt="a topographic map", background=True)

# `modality` is needed here too: video jobs and everything else's jobs live at
# different endpoints, and a job id does not say which it is.
done = client.media.wait(modality="image", job_id=started.job_id, on_progress=print)

At most one of url, b64 and data is set on an artifact, decided by the endpoint rather than by your request: url for the asynchronous modalities, b64 for images asked for as base64 and inline music, data for audio — POST /v1/audio/speech answers with an audio body, so there is nothing to link to. (data, not bytes, because bytes is a builtin.) Anything with no cross-modality meaning — revised_prompt, lyrics, resolution — is on result.raw, which holds the untouched per-modality value.

upscale routes on the source: pass image_url or video_url. The gateway refuses an image upscaler on the video route and vice versa, and the model slug does not say which it is, so the SDK asks rather than guesses.

What it gives up. No named-argument checking beyond modality and model: a misspelled duration_secnods reaches the wire, where the gateway ignores it and bills the model's default length, while videos.generate() refuses that call before it is sent because prompt and model are named there. When you know the modality at the call site, the named method is the better tool.

Images

image = client.images.generate(
    model="dall-e-3",
    prompt="an isometric dashboard, muted palette",
    size="1024x1024",
)
print(image.data[0]["url"])

Editing takes raw bytes — the base64 encoding happens here:

edited = client.images.edit(
    model="gpt-image-1",
    prompt="make the sky dramatic",
    image=open("room.png", "rb").read(),
    mask=open("sky-mask.png", "rb").read(),
)

Upscaling needs an upscale-modality model that accepts an image:

bigger = client.images.upscale(model="clarity-upscaler", image_url="https://…/small.png", scale=2)

Video

Generation is asynchronous by design. generate() submits and polls to completion:

video = client.videos.generate(
    model="veo-3",
    prompt="a drone shot over a coastline",
    duration=8,
    on_progress=lambda job: print(job.status, job.progress),
)

# A finished video carries its file on `result`, not on `data[]`.
print(video.result["url"])

To manage the job yourself:

job = client.videos.submit(model="veo-3", prompt="a reef at dawn")
status = client.videos.retrieve(job.id)

Note duration, not duration_seconds. Both reach the wire — **params is forwarded as given, which is what lets a parameter a model gained yesterday work today — and the gateway ignores the one it does not know, then bills the model's default length. The spelling is on VideoSubmitParams for exactly this reason.

Music

track = client.music.generate(
    model="suno-v5",
    prompt="lo-fi beat with a rainy-window feel",
    instrumental=True,
)
print(track.data[0]["url"], track.credits_used)

music.stream() yields tagged frames while the track renders. Every frame carries type, and a failure arrives as a frame rather than an exception — once SSE headers are sent the gateway cannot fall back to an HTTP error, so a caller that ignores type sees a successful, empty stream:

for event in client.music.stream(model="suno-v5", prompt="a rainy-window beat"):
    if event["type"] == "progress":
        print(event["status"], event["progress"])
    elif event["type"] == "completed":
        print(event["data"][0]["url"], event["credits_used"])
    elif event["type"] == "error":
        print("failed:", event["error"]["message"])

Audio

# Text to speech. `.audio` is the bytes, so you pick where they go; `.content_type`
# is what the server actually sent, and `.credits_used` is what the call cost.
speech = client.audio.speech.create(model="tts-1", voice="nova", input="Good morning.")
open("greeting.wav", "wb").write(speech.audio)
print(speech.content_type, speech.credits_used)

# Transcription. `srt`/`vtt`/`text` come back as the document STRING; anything
# else as a TranscriptionResult.
document = client.audio.transcriptions.create(
    model="whisper-1",
    file=open("meeting.mp3", "rb").read(),
    filename="meeting.mp3",
    response_format="vtt",
)

# Voice changing, stem separation, video-to-audio.
transformed = client.audio.transformations.create(model="demucs", audio_url="https://…/song.mp3")

audio.transcriptions.create is the one deferrable method whose return type does not include JobStatus: it maps the deferred job's own payload back into a TranscriptionResult, because the deliverable is text and the durable path stores it as text rather than as a file artifact. That matches the TypeScript SDK. It does accept background=True, where TypeScript deliberately does not — coherent, because JobDeferredError.job_id here carries what TypeScript had to encode in a return type.

3D

mesh = client.three_d.generate(model="trellis", image_url="https://…/chair.png")
print(mesh.data[0]["url"])

three_d, not threeD: Python names it the way Python names things. The TypeScript client's threeD is the same namespace.

Files

file = client.files.create(
    file=open("report.pdf", "rb").read(),
    filename="report.pdf",
    purpose="user_data",
)

page = client.files.list(purpose="user_data", limit=20)
if page.has_more:
    following = client.files.list(after=page.last_id)

download = client.files.content(file.id)
open("copy.bin", "wb").write(download.content)  # download.content_type says what it is
client.files.delete(file.id)

delete, not del: TypeScript needed del because delete is a reserved word there, and carrying that workaround into Python would import a problem Python does not have.

Uploads always send an Idempotency-Key, so a retry returns the file created the first time rather than storing a second copy.

Models and cost estimates

models = client.models.list(modality="image")
estimate = client.models.estimate("veo-3", duration=8)
print("about", estimate.credits, "credits")

tools = client.tools.list()
result = client.capabilities.run("web_search", input={"query": "latest pgvector release"})

An estimate is a quote, not a hold — nothing is reserved by asking.

Workflows

run = client.workflows.runs.create(
    workflow_id="wf_abc123",
    input={"topic": "quarterly summary"},
)
print(run.status, run.creditsUsed)

workflow_id, not pipeline_id: the request body says pipeline_id because the rename stopped at the HTTP boundary, and this SDK translates at its own boundary rather than making you type the old name. A raw pipeline_id still reaches the wire through **params — that is what forward compatibility costs — but passing both is refused before the request, because **params is spread last and the raw one would silently win, starting and billing a workflow other than the one you named.

To watch a run happen, runs.stream() yields one tagged event per step — fourteen shapes, all discriminated on type:

for event in client.workflows.runs.stream(workflow_id="wf_abc123", input={}):
    if event["type"] == "step.started":
        print("running", event["stepId"])
    elif event["type"] == "step.completed":
        print("done", event["stepId"], event["creditsUsed"])
    elif event["type"] == "pipeline.failed":
        print("failed", event["error"]["code"], event["error"]["message"])
    elif event["type"] == "unknown_event":
        # A gateway that adds a fifteenth event must not break this loop.
        print("new event", event["name"])

runs.stream() fixes mode="stream". A mode= arriving through **params is refused rather than forwarded: the gateway would run and settle the whole workflow synchronously, and the SDK would then read that JSON body as SSE and report a cut connection about a run you paid for in full.

A failed run does not have to be paid for twice — send its id as resume_from_run_id and it continues from the step that failed.

Deferred jobs

Media generation is submitted and awaited inside one request. If the gateway's own wait runs out it answers 504 with a job_id and keeps working — the work continues and is billed either way, so this SDK collects the finished result rather than failing:

# Either returns the image, or collects it after a deferral. You get an image.
image = client.images.generate(model="flux-pro", prompt="a topographic map")

Ask for the handle instead when you would rather poll yourself. background=True is accepted on ten methods, and the handle arrives as an exception, not as a return value — collect_deferred re-raises the gateway's own JobDeferredError instead of polling:

from infery import JobDeferredError

try:
    client.images.generate(model="flux-pro", prompt="a map", background=True)
except JobDeferredError as deferred:
    job = client.jobs.wait(deferred.job_id, interval=5.0, max_wait=600.0)
    print(job.data[0]["url"])

This is a real divergence from the TypeScript SDK, where background: true returns a result object carrying job_id. Python raises instead, and JobDeferredError.job_id is the carrier — which is also why audio.transcriptions.create can offer background=True here while the TypeScript method deliberately does not.

media.generate is the one exception, and the only method where a handle really is returned: it catches the deferral for you and answers with a MediaResult whose job_id is set and whose artifacts are empty.

started = client.media.generate(modality="image", model="flux-pro", prompt="a map", background=True)
done = client.media.wait(modality="image", job_id=started.job_id)

A JobStatus in a return union — ImageResponse | JobStatus and its siblings — is therefore always a collected result: the gateway deferred, the SDK polled to completion, and this is what it finished with. It is never a handle you have to poll yourself.

jobs.wait raises JobFailedError when the job reaches failed, and JobTimeoutError when max_wait elapses first. A JobTimeoutError is the client giving up on watching, not the server giving up on running: the job was never cancelled and is still billed, so poll it again later rather than starting a second one.

If artifacts_expired is True on a collected result, data is shorter than the job produced — the provider's links had already expired. None means "nothing known to be missing", not "nothing missing".

Errors

from infery import APIError, ConflictError, InsufficientCreditsError, RateLimitError

try:
    client.chat.completions.create(model="gpt-4o", messages=messages)
except InsufficientCreditsError:
    ...  # 402, or any status with `insufficient_credits`. Top up; do not retry.
except RateLimitError:
    ...  # 403 rate_limit_exceeded, a 60-second sliding window. Wait it out.
except ConflictError as err:
    if err.code == "upload_in_progress":
        ...  # Retry in a moment.
    raise
except APIError as err:
    print(err.status, err.code, err.request_id)

Each status this gateway produces maps to a subclass — AuthenticationError (401, and a 403 that is not a rate limit), InsufficientCreditsError (402), RateLimitError (403 rate_limit_exceeded), NotFoundError (404), ConflictError (409), and JobDeferredError for any status whose body carries a job_id. A status none of those covers lands on APIError itself, with status, code, message, request_id and the parsed body.

Everything the SDK raises descends from InferyError, so one except catches all of it — including the three that are not APIError at all: APIConnectionError/APITimeoutError (no response arrived), StreamTruncatedError (a stream ended without [DONE]), and JobFailedError/JobTimeoutError (a polled job).

Branch on err.code when the class is not specific enough. ConflictError's upload_in_progress means retry in a moment, while idempotency_in_progress means a billed run is already in flight and a retry could start a second one.

err.request_id is the handle support uses to attribute a charge — quote it when asking about a bill.

Cancelling and timeouts

Three numbers, all of them per client, all of them overridable:

client = Infery(
    api_key=os.environ["INFERY_API_KEY"],
    timeout=310.0,   # seconds, PER ATTEMPT
    max_wait=600.0,  # ceiling for collecting a deferred job
    max_retries=2,
)

timeout defaults to 310 seconds, deliberately above the gateway's own 300-second wait for a media generation so a slow one reaches the deferral handoff instead of being abandoned while it keeps billing. That is the right ceiling for generation and far too long for a catalogue read — construct a second client for short calls, or pass your own httpx client.

The deferrable methods take interval (default 5.0 s, the poll period) and max_wait (default 600.0 s) per call, so one long generation does not force the client-wide ceiling up.

Cancellation is the one place the two clients differ, because Python's are different mechanisms. On AsyncInfery, cancel the task — asyncio.timeout, task.cancel(), or leaving an async with block — and the request raises CancelledError at the next await point. On Infery there is no equivalent: the call returns when it returns, bounded by timeout and max_wait.

Either way, cancelling does not cancel work the gateway has already started, and does not refund it. An aborted generation is still billed.

Retries

Connection failures, 408/429/500/502/503 and 409 upload_in_progress are retried with a short backoff — 0.5 s then 1 s, max_retries attempts on top of the first, default 2 — but only when the request is a GET or hits one of the two endpoints that honour Idempotency-Key (POST /v1/files, POST /v1/workflows/runs). On every other billed POST — chat, image, video, audio generation — a 500 is not retried: the gateway collapses several distinct upstream failures, including ones that happen after your balance was debited, into the same generic 500, and retrying blind risks a second charge for a call that may already have succeeded.

A connection failure is no safer than a timeout, and gets the same rule. ECONNREFUSED looks like proof that nothing was delivered, but it is indistinguishable from a socket that reset after the request landed and generation had begun.

Two things are never retried whatever the method. 403 rate_limit_exceeded is a 60-second sliding window that counts refused requests too, so retrying inside it pushes your own recovery further out. And any response carrying a job_id means the work exists and is already billed — the SDK collects it rather than paying for a second one.

Using it with the OpenAI SDK instead

You can point the openai package at https://api.infery.ai/v1 and it will work for chat and embeddings. Past those it cannot reach the endpoints it has no methods for (video, music, 3D, upscaling, workflows), cannot collect a deferred result, and will not retry correctly, because rate limiting here answers 403 rather than 429.

The compatibility matrix answers this endpoint by endpoint.

Client options

Infery(
    api_key=os.environ["INFERY_API_KEY"],
    base_url="https://api.infery.ai/v1",  # default
    timeout=310.0,                        # seconds, per attempt
    max_retries=2,
    max_wait=600.0,                       # ceiling for collecting a deferred job
    http_client=None,                     # your own httpx.Client
)

api_key is required and is not read from the environment: a client built without one raises at construction rather than on the first request.

Pass http_client to reuse a connection pool, a proxy, a mounted transport or your own limits. A client given one never closes it — the caller owns what the caller made. A client that made its own closes it through close() (or a with block), and AsyncInfery through await aclose() (or async with).

aclose, not close, on the async side: closing an httpx.AsyncClient is a coroutine, and a method named close that has to be awaited is the shape that gets called without await and silently leaks the pool.

License

MIT