evalwatch 0.1.0

Python client for sending LLM traces to EvalWatch

evalwatch

Python client for sending LLM traces to EvalWatch — a SaaS observability and evaluation platform for LLM products.

Install

pip install evalwatch

Quick start

Get an API key from your EvalWatch dashboard (/settings), then instrument your LLM calls:

from evalwatch import EvalWatchClient

client = EvalWatchClient("https://your-evalwatch-instance.com", api_key="sk-...")

# 1. Manual
client.send_trace(
    prompt="What is the capital of France?",
    response="Paris.",
    latency_ms=120,
    model_name="gpt-4o",
    provider="openai",
    token_usage=30,
)

# 2. Context manager — measures latency automatically
with client.capture(prompt="...", model_name="gpt-4o", provider="openai") as trace:
    response = openai_client.chat.completions.create(...)
    trace.finish(response=response.choices[0].message.content, token_usage=response.usage.total_tokens)

# 3. Decorator — wraps any function whose first arg is the prompt
@client.watch(model_name="gpt-4o", provider="openai")
def call_llm(prompt: str) -> str:
    return openai_client.complete(prompt)

All three methods send traces to POST /api/v1/observe/traces and are visible in the Traces dashboard immediately.

License

MIT