ART SDK — AI Reliability Toolkit

Runtime observability and reliability for AI agents. Zero dependencies.

Install

pip install art-sdk
pip install art-sdk[openai]   # with OpenAI instrumentation

Quick Start

import art

# Evaluate any prompt/response pair
result = art.evaluate(
    prompt="What is the capital of France?",
    response="The capital of France is Paris."
)
print(result.confidence_score)     # 0–100
print(result.hallucination_score)  # 0–100 (lower is better)

# Auto-instrument OpenAI — every call gets evaluated
art.instrument_openai()
client = openai.OpenAI()
response = client.chat.completions.create(...)
# [ART] gpt-4o | confidence:82 hallucination:12 | $0.00023 | 340ms

# Group multi-step agent calls into a trace
with art.trace("session-abc"):
    r1 = client.chat.completions.create(...)  # step 0
    r2 = client.chat.completions.create(...)  # step 1

# Enforce a cost budget
with art.cost_guard(max_usd=0.05):
    response = client.chat.completions.create(...)
    # raises art.BudgetExceededError if over budget

Configure

art.init(
    pipeline="my-agent",          # tag your pipeline
    export_to="console",          # "console" | "art_cloud" | "none"
    art_cloud_url="http://...",   # ART Cloud endpoint (optional)
    max_usd_per_request=0.10,     # global cost cap (optional)
)

Release to PyPI

This repo publishes to PyPI with GitHub Actions via trusted publishing.

1. Bump version in pyproject.toml
2. Commit and push to main
3. Create and push a matching tag, e.g. v0.1.1
4. GitHub Actions builds and publishes to PyPI

First-time PyPI setup:

• In PyPI, add a trusted publisher (or pending publisher) for project art-sdk
• Owner: aixel-code
• Repository: art-sdk
• Workflow: publish-pypi.yml

Manual local fallback:

uv build
uv publish