graphn — Python SDK

The official Python SDK for Graphn. Author workflows, agents, functions, and knowledge bases, then run them — or import a custom model and chat through the OpenAI-compatible inference endpoint — from Python.

v0.2.x covers the full CLI-reachable Graphn API (control plane, inference, gateway, and storage). Python is the only first-party wrapped package; other languages generate from the public OpenAPI spec. See Scope below.

import graphn

with graphn.Client() as c:
    wf = c.workflows.create(name="qa", dsl="name: qa\nsteps: []")
    c.workflows.publish(wf.id)
    run = c.workflows.run(wf.id, input={"q": "..."})
    result = c.executions.wait(run.execution_id or "")
    hits = c.knowledgebases.search("kb_...", query="...")

Chat and TTS still go through the official openai package:

with graphn.Client() as c:
    resp = c.chat.completions.create(
        model="Qwen/Qwen3-0.6B",
        messages=[{"role": "user", "content": "Hello!"}],
    )
    print(resp.choices[0].message.content)

Install

pip install graphn

Requires Python 3.10+. Tested on 3.10, 3.11, 3.12, 3.13.

Authentication

The SDK reads credentials from the environment by default:

export GRAPHN_API_KEY=gn_...           # required
export GRAPHN_WORKSPACE_ID=ws_...      # required
export GRAPHN_BASE_URL=https://cp.graphn.ai          # optional
export GRAPHN_INFERENCE_URL=https://model.graphn.ai  # optional
export GRAPHN_GATEWAY_URL=https://gateway.graphn.ai  # optional
export GRAPHN_STORAGE_URL=https://storage.graphn.ai  # optional

Or pass them explicitly:

client = graphn.Client(api_key="gn_...", workspace_id="ws_...")

Get an API key from the Graphn dashboard.

Scope

What's in the box (v0.2.x)

Both graphn.Client and graphn.AsyncClient exist with identical APIs.

Not wrapped (generate from the spec, or wait)

These exist on the platform but are not first-class SDK resources in v0.2.x — file an issue on the SDK repo to vote on what you need next:

• Evals & datasets
• Guardrails (policy authoring)
• Voice agents / conversations
• Usage & billing beyond client.custom_models.gpu_hours()

Those endpoints can still be hit via raw HTTP using your gn_... API key. The OpenAPI 3.1 spec is mirrored at voltagepark/graphn-openapi and rendered at graphn.ai/api. Other languages generate from that spec (npx @hey-api/openapi-ts, oapi-codegen, openapi-generator-cli -g java).

The 80% recipe: import a model and chat with it

import graphn

with graphn.Client() as c:
    # 1. Import the model. Use a workspace secret for gated HF repos.
    model = c.custom_models.create(
        name="my-llama",
        huggingface_model_id="meta-llama/Llama-3.1-8B-Instruct",
        weight_source="huggingface",
        hf_token_secret_id="sec_...",  # optional, only for gated models
    )

    # 2. Wait for the deployment to be live.
    c.custom_models.wait_until_ready(model.id, timeout=1800)

    # 3. Chat. The first call will cold-start the model — the SDK
    #    transparently calls wake() and retries until it serves.
    resp = c.chat.completions.create(
        model=model.id,
        messages=[{"role": "user", "content": "Tell me a joke."}],
        wake_timeout=600,             # max time to wait for cold start
    )
    print(resp.choices[0].message.content)

Importing from S3

If your weights aren't on HuggingFace — fine-tunes, internal models, licensed checkpoints — import them straight from S3. Two flavors, both of which still require huggingface_model_id (see callout below).

huggingface_model_id is required for S3 imports too. It's the canonical identifier for the model — the name the inference endpoint advertises and the value you pass in model for chat completions. Use the upstream org/model-name your weights are based on (e.g. Qwen/Qwen3-0.6B, meta-llama/Llama-3.1-8B-Instruct). This is the same "Model ID" field the web UI requires for S3 imports. Omitting it raises graphn.ValidationError client-side; passing it but having a mismatched archive will surface as a deploy failure on the model record.

Presigned URL (no AWS credentials shared with Graphn):

model = c.custom_models.create(
    name="my-finetune",
    weight_source="s3_presigned",
    huggingface_model_id="meta-llama/Llama-3.1-8B-Instruct",
    s3_url="https://my-bucket.s3.amazonaws.com/llama-3.1-8b.tar.gz?X-Amz-Algorithm=...",
    gpu_count=1,
)

Package the weights as a single .tar.gz archive whose top level is the model directory (the same layout huggingface-cli download produces). Generate the URL with aws s3 presign s3://my-bucket/llama-3.1-8b.tar.gz or the AWS SDK; Graphn pulls weights through the URL on import. The URL only needs to be live for the import window (allow at least a few minutes for the download), not for the model's lifetime.

IAM role assumption (for buckets you control, longer-lived credentials):

model = c.custom_models.create(
    name="my-finetune",
    weight_source="s3_assume_role",
    huggingface_model_id="meta-llama/Llama-3.1-8B-Instruct",
    s3_url="s3://my-bucket/llama-3.1-8b.tar.gz",
    s3_role_arn="arn:aws:iam::123456789012:role/GraphnImport",
    gpu_count=1,
)

The role's trust policy must allow Graphn's importer principal to sts:AssumeRole; ask support for the principal ARN to put in your trust policy. Graphn re-assumes on every import / refresh, so rotating credentials underneath is safe.

Everything past the create call — wait_until_ready, chat completions, auto-wake, addressing by model.id — is identical regardless of weight source. See examples/import_from_s3.py for an end-to-end runnable script.

Streaming

stream = c.chat.completions.create(
    model=model.id,
    messages=[{"role": "user", "content": "Count to ten."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

Async

import asyncio
import graphn

async def main() -> None:
    async with graphn.AsyncClient() as c:
        async for m in c.custom_models.list():
            print(m.id, m.name, m.status)

        resp = await c.chat.completions.create(
            model="cm_abc123",
            messages=[{"role": "user", "content": "Hi!"}],
        )
        print(resp.choices[0].message.content)

asyncio.run(main())

Cold starts and auto-wake

Graphn custom models default to scale-to-zero: a model with no traffic for cooldown_seconds is descheduled, and the first request afterwards has to wait for the gateway to spin up a fresh replica (typically 60–600 seconds depending on weight size).

Without help, the first chat request after a cold period returns:

503 Service Unavailable: Model is scaled to zero and is now warming up.

The SDK detects this, calls POST /custom-models/{id}/wake to nudge the autoscaler, and retries with exponential backoff until the model serves or wake_timeout (default 180s) elapses. You don't have to do anything, but the knobs are there if you want them:

# Disable auto-wake — you handle the 503 yourself.
c.chat.completions.create(model=..., messages=[...], auto_wake=False)

# Give the warm-up more headroom (e.g. for large models).
c.chat.completions.create(model=..., messages=[...], wake_timeout=900)

See docs/cold-starts.md for the full story.

Drop-in for openai

The chat path is OpenAI-compatible all the way down — under the hood we delegate to the official openai Python SDK, configured against the Graphn gateway. So tools, structured outputs, multi-modal inputs, function calling, etc. all work out of the box.

If you already have OpenAI-shaped code and just want to point it at a Graphn model:

from openai import OpenAI

client = OpenAI(
    api_key="gn_...",
    base_url="https://model.graphn.ai/v1",
    default_headers={"X-Workspace-Id": "ws_..."},
)
resp = client.chat.completions.create(
    model="custom:cm_...",  # raw openai client => you type the prefix
    messages=[{"role": "user", "content": "Hello!"}],
)

The reason to use graphn.Client instead is everything around the chat call: lifecycle management, secrets, auto-wake, bare-cm_ addressing without the wire prefix, typed responses, and a stable URL contract.

More examples

See examples/ for runnable end-to-end scripts:

• examples/import_and_chat.py — full lifecycle (HuggingFace)
• examples/import_from_s3.py — S3 presigned + assume-role import
• examples/streaming.py — streaming chat
• examples/async_client.py — async usage
• examples/openai_compat.py — drop-in from openai

Configuration reference

Generating clients in other languages

The OpenAPI 3.1 spec is the source of truth. It's published at:

• GitHub — voltagepark/graphn-openapi
• Live HTML reference — graphn.ai/docs/api
• Direct download — https://cp.graphn.ai/openapi.yaml

Point your favorite generator at any of these. We test against openapi-generator 6.0+, openapi-python-client 0.21+, and oapi-codegen 2.0+.

Contributing

git clone https://github.com/voltagepark/graphn-sdk-python
cd graphn-sdk-python
python -m venv .venv && source .venv/bin/activate
pip install -e '.[dev]'

ruff check src tests
pytest

Regenerate the typed transport from the upstream spec after a spec change:

./scripts/regenerate.sh

See CHANGELOG.md for release notes.

License

Apache 2.0 — see LICENSE.