bolder-ai

Python SDK for BEval Studio by Bolder — log LLM/VLM/agent calls to your observability & evaluation dashboard.

Distribution name is bolder-ai. Import name is beval.

• Fire-and-forget, non-blocking (background thread queue)
• Works with raw calls, @trace decorator, or auto-instrumented OpenAI / Anthropic clients
• Zero hard deps beyond httpx

Install

pip install bolder-ai
# optional integrations
pip install 'bolder-ai[openai]'
pip install 'bolder-ai[anthropic]'

Then import beval in your code.

Requires Python 3.9+.

Configure

Set environment variables (or pass to beval.init(...)):

Quick start

1. Raw log

import beval

beval.init()  # reads env

beval.log(
    kind="llm",
    model_id="gpt-4o-mini",
    input="What is the capital of France?",
    output="Paris.",
    latency_ms=312,
    tokens_in=7,
    tokens_out=2,
)

2. Auto-wrap OpenAI

import beval
from openai import OpenAI

beval.init()
client = beval.wrap(OpenAI())

client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hi"}],
)

Automatically captures input messages, output, model, token counts, latency, errors. Detects image parts and logs as kind="vlm".

3. Auto-wrap Anthropic

import beval
from anthropic import Anthropic

beval.init()
client = beval.wrap(Anthropic())

client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=64,
    messages=[{"role": "user", "content": "Hi"}],
)

4. @beval.trace decorator

Wrap any function (sync or async) as an agent log:

@beval.trace
def run_agent(query: str) -> str:
    return ...

@beval.trace(name="tool:search", kind="agent")
async def search(q): ...

Captures args, return value, latency, and exceptions (logged with status="failure").

VLM / images

Pass image= to attach a base64 data URL (matches the dashboard's VLM preview):

beval.log(
    kind="vlm",
    model_id="gpt-4o",
    input="describe this",
    output="a cat",
    image=open("cat.png", "rb").read(),
    image_mime="image/png",
)

Redaction

Strip PII before send:

def redact(payload: dict) -> dict:
    if payload.get("input"):
        payload["input"] = scrub_pii(payload["input"])
    return payload

beval.init(redact=redact)

Lifecycle

• beval.flush(timeout=5.0) — wait for queued logs to drain
• beval.shutdown() — drain + close (runs automatically at interpreter exit)

Reliability

• Non-blocking: log() enqueues and returns immediately
• Network failures never raise — they're logged via the beval logger
• Drops on queue overflow (default capacity: 10,000)
• Retries transient errors (408/429/5xx) with exponential backoff

License

MIT