Calibrate Python Library

The Calibrate Python SDK provides convenient, typed access to the Calibrate agent evaluation platform API. Authenticate with an API key (sk_…) scoped to your organization to run agent tests and poll their results.

Documentation
Installation
Pre Requisites
Reference
Quick Start
Usage
Environments
Async Client
Exception Handling
Advanced

Documentation

API reference documentation is available here.

Installation

pip install calibrate-sdk

Pre-requisites

You need to have a Calibrate API key. You can get the API key from the UI.

Reference

A full reference for this library is available here.

Quick Start

from calibrate import Calibrate

client = Calibrate(api_key="sk_...")

# List the agents in your organization
agents = client.agents.list()

# Kick off a test run for an agent and poll for the result
run = client.agent_tests.run(agent_uuid="<agent-uuid>")
result = client.agent_tests.get_run(task_id=run.task_id)

Usage

Instantiate and use the client with the following:

from calibrate import Calibrate

client = Calibrate(
    api_key="<value>",
)

client.agents.verify_connection(
    agent_uuid="f47ac10b-58cc-4372-a567-0e02b2c3d479",
)

Environments

This SDK allows you to configure different environments for API requests.

from calibrate import Calibrate
from calibrate.environment import CalibrateEnvironment

client = Calibrate(
    environment=CalibrateEnvironment.DEFAULT,
)

Async Client

The SDK also exports an async client so that you can make non-blocking calls to our API. Note that if you are constructing an Async httpx client class to pass into this client, use httpx.AsyncClient() instead of httpx.Client() (e.g. for the httpx_client parameter of this client).

import asyncio

from calibrate import AsyncCalibrate

client = AsyncCalibrate(
    api_key="<value>",
)


async def main() -> None:
    await client.agents.verify_connection(
        agent_uuid="f47ac10b-58cc-4372-a567-0e02b2c3d479",
    )


asyncio.run(main())

Exception Handling

When the API returns a non-success status code (4xx or 5xx response), a subclass of the following error will be thrown.

from calibrate.core.api_error import ApiError

try:
    client.agents.verify_connection(...)
except ApiError as e:
    print(e.status_code)
    print(e.body)

Advanced

Access Raw Response Data

The SDK provides access to raw response data, including headers, through the .with_raw_response property. The .with_raw_response property returns a "raw" client that can be used to access the .headers and .data attributes.

from calibrate import Calibrate

client = Calibrate(...)
response = client.agents.with_raw_response.verify_connection(...)
print(response.headers)  # access the response headers
print(response.status_code)  # access the response status code
print(response.data)  # access the underlying object

Retries

The SDK is instrumented with automatic retries with exponential backoff. A request will be retried as long as the request is deemed retryable and the number of retry attempts has not grown larger than the configured retry limit (default: 2).

Which status codes are retried depends on the retryStatusCodes generator configuration:

legacy (current default): retries on

408 (Timeout)
409 (Conflict)
429 (Too Many Requests)
5XX (All server errors, including 500)

recommended: retries on

408 (Timeout)
409 (Conflict)
429 (Too Many Requests)
502 (Bad Gateway)
503 (Service Unavailable)
504 (Gateway Timeout)

Use the max_retries request option to configure this behavior.

client.agents.verify_connection(..., request_options={
    "max_retries": 1
})

Timeouts

The SDK defaults to a 60 second timeout. You can configure this with a timeout option at the client or request level.

from calibrate import Calibrate

client = Calibrate(..., timeout=20.0)

# Override timeout for a specific method
client.agents.verify_connection(..., request_options={
    "timeout_in_seconds": 1
})

Custom Client

You can override the httpx client to customize it for your use-case. Some common use-cases include support for proxies and transports.

import httpx
from calibrate import Calibrate

client = Calibrate(
    ...,
    httpx_client=httpx.Client(
        proxy="http://my.test.proxy.example.com",
        transport=httpx.HTTPTransport(local_address="0.0.0.0"),
    ),
)

Metadata

Release files for calibrate-sdk 0.0.65

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for calibrate-sdk 0.0.65
File	Size	Uploaded
calibrate_sdk-0.0.65.tar.gz	89.6 kB	Sep 20, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for calibrate-sdk 0.0.65
File	Interpreter	ABI	Platform	Reset
calibrate_sdk-0.0.65-py3-none-any.whl 172.7 kB Sep 20, 2026	Python 3	none	any	Details

Total release size: 262.3 kB

Release files / calibrate_sdk-0.0.65.tar.gz

Download URL	calibrate_sdk-0.0.65.tar.gz
Size	89.6 kB
Tags	Source
SHA-256 checksum How to use checksums	`ecf9691dcfb4294d6a2fb2d89bd7af7ab3fe5e70357c664c592e8df9e09bd50d`
BLAKE2b-256 checksum How to use checksums	`1847d7bdd554d339a1ae9f8c5c9244796997c222e2c6111f39466eddaf504d31`
Upload date	Sep 20, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.5.1 CPython/3.10.21 Linux/6.17.0-1022-azure`

Release files / calibrate_sdk-0.0.65-py3-none-any.whl

Download URL	calibrate_sdk-0.0.65-py3-none-any.whl
Size	172.7 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`d95e732a5d0e94ad9ba42ead3af0752e7da7601c979f5cc1fda75d2d1d19a156`
BLAKE2b-256 checksum How to use checksums	`545f4a8c891e82de9b72abbfb46724291dbad55ffc198b46297a905c625e1b09`
Upload date	Sep 20, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.5.1 CPython/3.10.21 Linux/6.17.0-1022-azure`