belvedir

The Belvedir SDK for Python: auto-instrument your AI agent for recursive self-improvement.

The Python counterpart of belvedir on npm: same concepts (initialize, sessions, tasks, flush), Python idioms (context managers instead of callbacks).

Already using belvedir-loop? Nothing breaks — belvedir-loop@0.4.0 stays on PyPI and keeps working. It just won't get new releases. import belvedir_loop also still works after you switch, with no deprecation warning: this package ships that module too, and belvedir re-exports it.

Switching from belvedir-loop, remove it first:

pip uninstall belvedir-loop && pip install belvedir

Installing belvedir alongside an existing belvedir-loop<=0.4.0 leaves both distributions claiming the same belvedir_loop/ files. Everything imports and runs, but a later pip uninstall belvedir then deletes files the older belvedir-loop still records as its own, and import belvedir_loop breaks until you reinstall. pip install --force-reinstall belvedir repairs it.

Install

pip install belvedir

Quick Start

Call initialize() once at process startup, before your agent starts making LLM calls. Instrumentation (via OpenLLMetry) patches the installed LLM client libraries at the class level, so clients created earlier are still traced, but initializing first is the safe default:

import os
import belvedir as loop

loop.initialize(
    api_key=os.environ["BELVEDIR_API_KEY"],
    app_name="my-agent",
)

There is no instrumentModules equivalent: Python has no bundler, so auto-instrumentation always sees your installed anthropic / openai / LangChain / etc. packages.

Raw HTTP calls are covered too (0.6.0): if your agent calls an OpenAI-compatible endpoint (the Belvedir router, OpenRouter, vLLM, ...) with plain requests or httpx instead of a client SDK, initialize() captures those calls as full LLM spans — streamed (SSE) responses included, with the assistant's reply accumulated in the background while your app reads the stream. Opt out with instrument_http=False.

Sessions: link traces so Belvedir can find tasks

This is what powers Tasks & Groups. Wrap each agent run in session so every LLM and tool span is linked under one session id. Optionally wrap distinct units of work in task() for sharper task boundaries:

with loop.session(session_id=chat_id, user_id=user.id):
    with loop.task("send_email"):
        agent.run("email danny the report")

Both are plain context managers; they propagate across await boundaries, so they work inside async def bodies too. Without session you still get raw traces, but they can't be grouped into sessions or segmented into tasks. (with_session is an alias for session, mirroring the JS SDK's withSession.)

Short-lived processes: flush before exiting

Spans are exported in batches. Scripts, batch jobs, and serverless handlers can exit before the batch ships; call flush() before returning:

loop.flush()

Or pass disable_batch=True to initialize() to export every span immediately (useful for local testing).

Report outcomes (ground truth)

When your app knows whether a session actually worked (the order shipped, the test passed, the user accepted), tell Belvedir. Reported outcomes beat the platform's model-judged labels: a "fail" keeps every task in that session out of training data.

loop.flush()  # the session must have arrived first
loop.report_outcome(chat_id, "success" if order_shipped else "fail")

report_outcome never raises and returns whether the report was accepted (False with a warning if Belvedir hasn't seen the session yet; retry after flush(), or report later from a webhook or job).

Import traces from other platforms

Already tracing with LangSmith? Import that history instead of waiting for fresh traffic. Imported traces go through the normal ingest path, so they form sessions, tasks, and groups, and count toward training, attributed to the API key you pass, like live traffic.

# list what would be imported
python -m belvedir.importers.langsmith \
    --langsmith-key lsv2_pt_... --project my-project \
    --belvedir-key bv_live_... --list

# import the 20 most recent conversations
python -m belvedir.importers.langsmith \
    --langsmith-key lsv2_pt_... --project my-project \
    --belvedir-key bv_live_... --limit 20

Or from Python:

from belvedir.importers.langsmith import LangSmithImporter

imp = LangSmithImporter(api_key="lsv2_pt_...", project="my-project")
conversations = imp.list_conversations(limit=20)
imp.import_conversations(
    [c.id for c in conversations],
    belvedir_api_key="bv_live_...",
)

Conversations are LangSmith traces grouped by their session_id / conversation_id / thread_id metadata; traces without any are imported individually. LLM runs keep their message history, tool runs their names, errors their status. Tasks appear ~30s after an import finishes. Re-importing the same conversation duplicates its spans; import once.

From a file (CSV / JSONL / JSON)

For datasets that never went through an observability platform (exported chat logs, fine-tuning files, spreadsheets):

# preview what the file parses into
python -m belvedir.importers.files \
    --file conversations.jsonl --belvedir-key bv_live_... --list

# import it
python -m belvedir.importers.files \
    --file conversations.jsonl --belvedir-key bv_live_...

Shapes are detected automatically. JSONL: one conversation per line, {"messages": [...]} (OpenAI chat/fine-tune format; LangChain and Anthropic message encodings work too), a bare message list, or a single {"input": ..., "output": ...} exchange (prompt/response and question/answer too); an id/session_id field names the conversation and lines sharing one merge into it. CSV: role + content columns give one message per row (grouped by a session_id-style column, or the whole file as one conversation), or an input/output-style column pair gives one exchange per row. JSON: a list of conversations or one {"messages": ...} object. Conversations without timestamps are stamped relative to import time. Same caveat as above: import a file once.

Configuration

API

Supported Providers

Anything OpenLLMetry instruments, including OpenAI, Anthropic, Cohere, Azure OpenAI, Amazon Bedrock, Google Vertex AI, Replicate, HuggingFace, and frameworks like LangChain and LlamaIndex.

Local Development

If running Belvedir locally:

loop.initialize(
    api_key="your-key",
    base_url="http://localhost:3000",
    app_name="my-app",
    disable_batch=True,  # send spans immediately
)

Troubleshooting

Traces arrive but no tasks or groups form. Work must run inside session. Tasks appear ~30s after a session goes quiet.

No traces at all. base_url must be a host that serves the ingest API (https://platform.belvedir.ai, the default; https://belvedir.ai still works; the pre-rename platform.fractalresearch.ai is retired); any other host 404s and spans drop silently. For short-lived processes, make sure flush() runs before exit. Initialization and export failures never raise; they log warnings under the belvedir.loop logger, so check your logs.