Xinference Cloud Python SDK

Official Python SDK for the Xinference Cloud management APIs: browse the deployable model catalog, create and manage model deployments, and issue inference API keys — all from Python.

Installation

pip install xinference-cloud

Requires Python 3.11 or later. The only runtime dependency is httpx.

Authentication

The SDK authenticates with an SDK management credential scoped to your organization. You can obtain one in two ways:

• Console UI — the SDK Credentials page in the Xinference Cloud console (in development, rolling out soon).

• Management API — while signed in to the console, create one via POST /api/v1/organization/sdk-credentials:

  // Run in your browser DevTools console while signed in
  const res = await fetch("/api/v1/organization/sdk-credentials", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ name: "my-sdk-credential" }),
  });
  console.log((await res.json()).key);
  

The credential key is shown only once at creation time — store it securely. If you lose it, revoke the credential and create a new one.

Quickstart

from xinference_cloud import XinferenceCloud

client = XinferenceCloud(
    base_url="https://<your-region>.cloud.xinference.co",
    credential="<your-sdk-credential>",
)

with client:
    # Browse the deployable model catalog
    models = client.deployable_models.list(enabled_only=True)

    # Deploy a model and wait until it is running
    deployment = client.deployments.create(model_name="qwen2.5-instruct")
    deployment = client.deployments.wait(deployment["id"], timeout=600)

    # Issue an inference API key bound to the deployment
    api_key = client.inference_api_keys.create(
        name="production",
        deployment_ids=[deployment["id"]],
    )

Instead of passing arguments explicitly, you can set the XINFERENCE_CLOUD_BASE_URL and XINFERENCE_CLOUD_CREDENTIAL environment variables and construct the client with XinferenceCloud.from_env().

Resources

deployments.wait() polls a deployment until it reaches a target status (running by default), raises RuntimeError if it enters a failure status, and raises TimeoutError on timeout.

Error handling

API errors raise xinference_cloud.APIStatusError, which carries the HTTP status_code, the server error code (when provided), and the raw httpx.Response:

from xinference_cloud import APIStatusError

try:
    client.deployments.get("unknown-id")
except APIStatusError as exc:
    print(exc.status_code, exc.code, str(exc))

License

Apache-2.0