ashr-labs 0.1.0

Python SDK for the Ashr Labs voice agent testing platform

Ashr Labs Python SDK

A Python client library for interacting with the Ashr Labs API.

Documentation

• Quick Start Guide
• Installation
• Authentication
• API Reference
• Error Handling
• Examples

Installation

pip install ashr-labs

Or install from source:

cd sdk
pip install -e .

Quick Start

from ashr_labs import AshrLabsClient

# Initialize the client with your API key
client = AshrLabsClient(
    api_key="tp_your_api_key_here",
    base_url="https://api.ashr.io/testing-platform-api"
)

# Initialize session and get user/tenant info
session = client.init()
print(f"Logged in as: {session['user']['email']}")
print(f"Tenant: {session['tenant']['name']}")

# List datasets
response = client.list_datasets(tenant_id=1)
for dataset in response["datasets"]:
    print(f"Dataset: {dataset['name']} (ID: {dataset['id']})")

# Get a specific dataset with signed URLs for media files
dataset = client.get_dataset(
    dataset_id=42,
    include_signed_urls=True,
    url_expires_seconds=3600
)

# Build a test run incrementally as the agent executes
from ashr_labs import RunBuilder

run_builder = RunBuilder()
run_builder.start()

test = run_builder.add_test("bank_analysis")
test.start()
test.add_user_text(text="Analyze this statement", description="User prompt")
test.add_tool_call(
    expected={"tool_name": "extract_pdf", "arguments": {"file": "a.pdf"}},
    actual={"tool_name": "extract_pdf", "arguments": {"file": "a.pdf"}},
    match_status="exact",
)
test.add_agent_response(
    expected_response={"summary": "Expected..."},
    actual_response={"summary": "Actual..."},
    match_status="similar",
    semantic_similarity=0.89,
)
test.complete()
run_builder.complete()

# Deploy the run to the API
run_builder.deploy(client, tenant_id=1, dataset_id=42)

# Create a request
request = client.create_request(
    tenant_id=1,
    requestor_id=5,
    request_name="Generate Audio",
    request={
        "text": "Hello world",
        "voice": "alloy"
    }
)
print(f"Created request: {request['id']}")

Available Methods

Datasets

Method	Description
get_dataset(dataset_id, include_signed_urls, url_expires_seconds)	Get a dataset by ID
list_datasets(tenant_id, limit, offset, include_signed_urls)	List datasets for a tenant

Runs

Method	Description
create_run(tenant_id, dataset_id, result, runner_id)	Create a new test run
get_run(run_id)	Get a run by ID
list_runs(tenant_id, dataset_id, limit, offset)	List runs
delete_run(run_id)	Delete a run

RunBuilder

Method	Description
RunBuilder()	Create a new run builder
run.start()	Mark the run as started
run.add_test(test_id)	Add a test and get a TestBuilder
run.complete(status)	Mark the run as completed
run.build()	Serialize to a result dict
run.deploy(client, tenant_id, dataset_id)	Build and submit via the API

TestBuilder

Method	Description
test.start()	Mark the test as started
test.add_user_file(file_path, description)	Record a user file upload
test.add_user_text(text, description)	Record a user text input
test.add_tool_call(expected, actual, match_status)	Record an agent tool call
test.add_agent_response(expected_response, actual_response, match_status)	Record an agent response
test.complete(status)	Mark the test as completed

Requests

Method	Description
create_request(tenant_id, requestor_id, request_name, request, request_input_schema)	Create a new request
get_request(request_id)	Get a request by ID
list_requests(tenant_id, status, limit, offset)	List requests for a tenant

API Keys

Method	Description
list_api_keys(include_inactive)	List API keys for your tenant
revoke_api_key(api_key_id)	Revoke an API key

Session

Method	Description
init()	Initialize session and get user/tenant info

Utility

Method	Description
health_check()	Check if the API is reachable

Error Handling

The SDK raises specific exceptions for different error types:

from ashr_labs import (
    AshrLabsClient,
    AuthenticationError,
    AuthorizationError,
    NotFoundError,
    ValidationError,
    RateLimitError,
    ServerError,
)

client = AshrLabsClient(api_key="tp_...", base_url="https://...")

try:
    dataset = client.get_dataset(dataset_id=999)
except AuthenticationError:
    print("Invalid API key")
except AuthorizationError:
    print("You don't have access to this resource")
except NotFoundError:
    print("Dataset not found")
except ValidationError as e:
    print(f"Invalid request: {e.message}")
except RateLimitError:
    print("Rate limit exceeded, please retry later")
except ServerError:
    print("Server error, please try again")

API Key Permissions

API keys have access to a limited set of operations:

• Datasets: Read-only access (get, list)
• Runs: Full access (create, get, list, delete)
• Requests: Create and read access
• API Keys: List and revoke (cannot create new keys via API)

To create new API keys, use the web interface with OAuth authentication.

Configuration

Timeout

You can configure the request timeout (default: 30 seconds):

client = AshrLabsClient(
    api_key="tp_...",
    base_url="https://...",
    timeout=60  # 60 seconds
)

Requirements

• Python 3.10+
• No external dependencies (uses only standard library)

License

MIT