TraceroAI Python SDK
Send RAG traces to TraceroAI — a RAG observability and evaluation platform. Instrument any RAG pipeline (LangChain, LlamaIndex, or your own) and every answer becomes a debuggable trace.
Install
pip install traceroai
Usage
Context manager (recommended)
Times the block and sends the trace automatically:
from traceroai import TraceroClient
client = TraceroClient(
base_url="https://traceroai.onrender.com",
api_key="your_project_key",
)
with client.trace("How long does a refund take?") as t:
t.log_retrieval(chunks, strategy="hybrid", config={"final_top_k": 3})
t.log_prompt(prompt_text, version="grounded_v1")
t.log_generation(
answer,
model="gpt-4o-mini",
temperature=0,
parameters={"top_p": 1, "max_tokens": 512}, # any tunable knobs
prompt_tokens=1200, completion_tokens=80, # -> server computes cost
)
print(t.trace_id)
Read a trace back (server-computed diagnosis + evaluations):
trace = client.get_trace(t.trace_id)
trace["diagnosis"]["label"] # e.g. "healthy_answer"
trace["generation"]["usage"] # tokens + cost_usd
Decorator
For a function that returns (answer, chunks):
@client.traced(model="gpt-4o-mini", strategy="hybrid")
def answer(query: str):
chunks = retrieve(query)
return generate(query, chunks), chunks
answer("What is the maximum file upload size?") # traced automatically
Low-level
client.log_trace(
query={"original": question},
retrieval={"strategy": "hybrid", "chunks": chunks},
generation={"model": "gpt-4o-mini", "answer": answer},
)
Authentication (multi-tenant)
Pass your project API key; the server attributes traces to your project:
client = TraceroClient(base_url="https://traceroai.onrender.com", api_key="your_project_key")
Self-healing recovery (optional)
pip install "traceroai[recovery]"
RecoveryAgent (built on LangGraph) retries the RAG stage that TraceroAI diagnoses as
broken — re-retrieving on a retrieval miss, re-generating with a stricter prompt on an
unsupported claim or a wrong answer — until the answer is healthy or it escalates to
review. You supply your own retrieve/generate; every attempt is traced.
from traceroai.recovery import RecoveryAgent
agent = RecoveryAgent(client, retrieve=my_retrieve, generate=my_generate, max_attempts=3)
result = agent.run("How long does a refund take?")
# result["answer"], result["diagnosis"], result["attempts"], result["trace_ids"]
generate may return just the answer, or (answer, {"prompt_tokens": ..., "completion_tokens": ...}) —
return the token counts and each recovery attempt's trace gets a server-computed cost.
Routing is judge-driven (v0.4.0+). Each attempt is evaluated by the server's LLM judge synchronously, so recovery routes on a judge-quality diagnosis — not the cheap deterministic quick eval — without polling. If the judge is unavailable, it falls back to the quick diagnosis so recovery still progresses (it never hangs or breaks). This needs a TraceroAI server with deep eval configured; against an older server it transparently falls back to the quick-eval routing.
Experiment evaluation
A/B-test pipeline configs against a labeled dataset. Bring your own
retrieve/generate and cases; each answer is graded by TraceroAI's server-side
judge, the best variant is recommended, and the run shows up on your dashboard.
from traceroai.eval import run_experiment, Case, Variant
run_experiment(
client=client,
dataset=[Case("c1", "How long does a refund take?", "5-7 business days.")],
retrieve=my_retrieve, # (query, top_k) -> list[chunk dict]
generate=my_generate, # (query, context) -> answer str
variants=[Variant("k3", "top_k=3", top_k=3), Variant("k5", "top_k=5", top_k=5)],
project_id="my-app",
)
Telemetry is best-effort
If the API is unreachable, the SDK warns and continues — it never breaks your app or masks your own exceptions. Evaluations, diagnosis, and cost are computed server-side (the server is the source of truth).
Metadata
Release files for traceroai 0.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| traceroai-0.4.0.tar.gz | 17.5 kB | Without provenance | Details for traceroai-0.4.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| traceroai-0.4.0-py3-none-any.whl | Python 3 | none | any | Details for traceroai-0.4.0-py3-none-any.whl |
Total release size: 32.6 kB
Release files / traceroai-0.4.0.tar.gz
| Download URL | traceroai-0.4.0.tar.gz |
|---|---|
| Size | 17.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d4733de13aa3cf5b866ff64c4d872b50fbf79bcff992fadd245db895703f3687
|
|
BLAKE2b-256 checksum How to use checksums |
7799b5957e2c2f6f14c701b709e2800a05e5cce35382d33ed7e296fbfa5d69ea
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.1
|
Release files / traceroai-0.4.0-py3-none-any.whl
| Download URL | traceroai-0.4.0-py3-none-any.whl |
|---|---|
| Size | 15.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
78a92a1545f48f450346d1ca800a6c2867d545a4974cb9fd29e184ebc9064fac
|
|
BLAKE2b-256 checksum How to use checksums |
d12edd5acf02b86b7b332e69093620989f8eda520f634ebcb897c117f236c4b0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.1
|
Security for traceroai 0.4.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.