belvedir-loop

Belvedir Loop SDK for Python — auto-instrument your AI agent for recursive self-improvement.

The Python counterpart of @belvedir/loop: same concepts (initialize, sessions, tasks, flush), Python idioms (context managers instead of callbacks).

Install

pip install belvedir-loop

Quick Start

Call initialize() once at process startup — before your agent starts making LLM calls. Instrumentation (via OpenLLMetry) patches the installed LLM client libraries at the class level, so clients created earlier are still traced, but initializing first is the safe default:

import os
import belvedir_loop as loop

loop.initialize(
    api_key=os.environ["BELVEDIR_API_KEY"],
    app_name="my-agent",
)

There is no instrumentModules equivalent — Python has no bundler, so auto-instrumentation always sees your installed anthropic / openai / LangChain / etc. packages.

Sessions — link traces so Belvedir can find tasks

This is what powers Tasks & Groups. Wrap each agent run in session so every LLM and tool span is linked under one session id. Optionally wrap distinct units of work in task() for sharper task boundaries:

with loop.session(session_id=chat_id, user_id=user.id):
    with loop.task("send_email"):
        agent.run("email danny the report")

Both are plain context managers; they propagate across await boundaries, so they work inside async def bodies too. Without session you still get raw traces, but they can't be grouped into sessions or segmented into tasks. (with_session is an alias for session, mirroring the JS SDK's withSession.)

Short-lived processes — flush before exiting

Spans are exported in batches. Scripts, batch jobs, and serverless handlers can exit before the batch ships — call flush() before returning:

loop.flush()

Or pass disable_batch=True to initialize() to export every span immediately (useful for local testing).

Report outcomes (ground truth)

When your app knows whether a session actually worked — the order shipped, the test passed, the user accepted — tell Belvedir. Reported outcomes beat the platform's model-judged labels: a "fail" keeps every task in that session out of training data.

loop.flush()  # the session must have arrived first
loop.report_outcome(chat_id, "success" if order_shipped else "fail")

report_outcome never raises and returns whether the report was accepted (False with a warning if Belvedir hasn't seen the session yet — retry after flush(), or report later from a webhook or job).

Import traces from other platforms

Already tracing with LangSmith? Import that history instead of waiting for fresh traffic. Imported traces go through the normal ingest path, so they form sessions, tasks, and groups, and count toward training — attributed to the API key you pass, like live traffic.

# list what would be imported
python -m belvedir_loop.importers.langsmith \
    --langsmith-key lsv2_pt_... --project my-project \
    --belvedir-key bv_live_... --list

# import the 20 most recent conversations
python -m belvedir_loop.importers.langsmith \
    --langsmith-key lsv2_pt_... --project my-project \
    --belvedir-key bv_live_... --limit 20

Or from Python:

from belvedir_loop.importers.langsmith import LangSmithImporter

imp = LangSmithImporter(api_key="lsv2_pt_...", project="my-project")
conversations = imp.list_conversations(limit=20)
imp.import_conversations(
    [c.id for c in conversations],
    belvedir_api_key="bv_live_...",
)

Conversations are LangSmith traces grouped by their session_id / conversation_id / thread_id metadata; traces without any are imported individually. LLM runs keep their message history, tool runs their names, errors their status. Tasks appear ~30s after an import finishes. Re-importing the same conversation duplicates its spans — import once.

From a file (CSV / JSONL / JSON)

For datasets that never went through an observability platform — exported chat logs, fine-tuning files, spreadsheets:

# preview what the file parses into
python -m belvedir_loop.importers.files \
    --file conversations.jsonl --belvedir-key bv_live_... --list

# import it
python -m belvedir_loop.importers.files \
    --file conversations.jsonl --belvedir-key bv_live_...

Shapes are detected automatically. JSONL: one conversation per line — {"messages": [...]} (OpenAI chat/fine-tune format; LangChain and Anthropic message encodings work too), a bare message list, or a single {"input": ..., "output": ...} exchange (prompt/response and question/answer too); an id/session_id field names the conversation and lines sharing one merge into it. CSV: role + content columns give one message per row (grouped by a session_id-style column, or the whole file as one conversation), or an input/output-style column pair gives one exchange per row. JSON: a list of conversations or one {"messages": ...} object. Conversations without timestamps are stamped relative to import time. Same caveat as above: import a file once.

Configuration

Option	Type	Default	Description
api_key	str	required	Your Belvedir API key (fr_live_...)
base_url	str	https://platform.belvedir.ai	Belvedir platform URL
app_name	str	belvedir-loop-app	Your application name
disable_batch	bool	False	Export each span immediately instead of batching

API

Function	Description
initialize(...)	Patches your LLM clients and starts exporting spans. Call once, at startup.
session(session_id, user_id=None, metadata=None)	Context manager: every span produced inside is linked under the session id. Required for Tasks & Groups. with_session is an alias.
task(name)	Context manager: optional boundary hint wrapping one unit of agent work in a named span, so segmentation knows where a task starts and ends.
flush()	Forces pending spans to export. Call before a short-lived process exits. Never raises.

Supported Providers

Anything OpenLLMetry instruments, including OpenAI, Anthropic, Cohere, Azure OpenAI, Amazon Bedrock, Google Vertex AI, Replicate, HuggingFace, and frameworks like LangChain and LlamaIndex.

Local Development

If running Belvedir locally:

loop.initialize(
    api_key="your-key",
    base_url="http://localhost:3000",
    app_name="my-app",
    disable_batch=True,  # send spans immediately
)

Troubleshooting

Traces arrive but no tasks or groups form. Work must run inside session. Tasks appear ~30s after a session goes quiet.

No traces at all. base_url must be a host that serves the ingest API (https://platform.belvedir.ai, the default; https://belvedir.ai still works; the pre-rename platform.fractalresearch.ai is retired) — any other host 404s and spans drop silently. For short-lived processes, make sure flush() runs before exit. Initialization and export failures never raise; they log warnings under the belvedir.loop logger, so check your logs.