dobby-collector 0.9.0

Telemetry collector for AI agents — stream runs, tools, LLM calls, and chains to the Dobby AI Control Plane for governance

dobby-collector

Telemetry collector for AI agents — stream runs, tools, LLM calls, and chains to the Dobby AI Control Plane for governance and compliance.

dobby-collector is the customer-side Python package for sending telemetry from any AI agent — LangChain, CrewAI, AutoGen, plain OpenAI/Anthropic SDKs, or custom code — to Dobby's governance plane. Every captured run gets evaluated by the Policy Scanner against your imported Compliance Packs (SOC 2, GDPR, EU AI Act, etc.) and surfaces violations in the Dobby dashboard.

Install

pip install dobby-collector

Quickstart

from dobby_collector import init, track, span, start_run, end_run

# 1. Initialize once at agent startup
init(
    api_key="dsdk_...",       # Generate at /dashboard/workloads/connect/python-sdk
    connector_id="wc_...",       # Same wizard hands you the connector ID
    framework="langchain",        # or 'crewai' | 'autogen' | omit to auto-detect
)

# 2. Decorate tool functions
@track(name="search_database", kind="tool")
def search_db(query: str) -> list:
    return db.execute(query)

# 3. Use spans for fine-grained capture
with span("retrieval", kind="tool", inputs={"query": "find AI startups"}):
    docs = retriever.invoke("find AI startups")

# 4. Wrap agent invocations in start/end_run
run = start_run(name="weekly_report", inputs={"week": "2026-W19"})
try:
    output = my_agent.run("Generate the weekly report")
    end_run(run, outputs={"report": output}, status="success")
except Exception as e:
    end_run(run, error=str(e), status="error")
    raise

The SDK runs a background thread that flushes events every 10 seconds (or immediately on terminal run.completed / run.failed events). Telemetry never blocks your agent — buffer overflow drops the oldest events silently.

What gets captured

What	When	Captured fields
Run boundaries	start_run / end_run	inputs, outputs, status, duration
Tool calls	@track(kind="tool") or span(kind="tool")	tool name, args, output, duration
LLM calls	LangChain / Gemini (google-genai) auto-instrument, or @track(kind="llm")	model, prompt, completion, tokens, latency
Custom spans	@track() / span()	name, inputs, outputs, duration

Config reference

init(
    api_key="dsdk_...",              # REQUIRED — or set DOBBY_API_KEY env var
    connector_id="wc_...",              # REQUIRED — or set DOBBY_CONNECTOR_ID env var
    base_url="https://dobby-ai.com",    # Override for self-hosted; or DOBBY_BASE_URL env
    flush_interval_seconds=10.0,        # How often the sender thread flushes
    max_buffer_events=10_000,           # Ring buffer cap (oldest dropped on overflow)
    framework="auto",                   # Hint or pin: 'langchain' | 'crewai' | 'autogen'
    host_fingerprint=None,              # Optional hostname/container ID for multi-replica
    exclude_fields=[],                  # Fields to scrub from `data` payloads (Phase 2b)
    agent_anchor={"agent_key": "..."},  # Agent identity — see "Agent identity" below
)

Environment variables

The SDK reads these as fallbacks for init() args:

Env var	Default	Purpose
DOBBY_API_KEY	—	Connector bearer token (dsdk_*) — minted by the wizard
DOBBY_CONNECTOR_ID	—	Workload connector ID (wc_*)
DOBBY_BASE_URL	https://dobby-ai.com	Dobby control-plane URL
DOBBY_AGENT_KEY	—	Stable agent identity — required for attestation, see below
DOBBY_AGENT_VERSION	—	Agent version, e.g. 2.3.1 (recorded, never a merge key)
DOBBY_ENV	—	Deployment env, e.g. prod / staging
DOBBY_AGENT_OWNER	—	Owning team, e.g. risk-team
DOBBY_DLQ_PATH	~/.dobby-collector/dlq.sqlite	Dead-letter-queue file
DOBBY_DLQ_DISABLED	—	Set to 1 to disable the DLQ (tests / read-only FS)

Set them in your deployment config and skip the corresponding init() args.

Agent identity (attestation)

Declare a stable agent_key, or this agent can never be attested. It is the most-missed setup step, and nothing in the collector will fail without it.

dobby_collector.init(
    api_key="dsdk_...",
    connector_id="wc_...",
    agent_anchor={
        "agent_key": "kyc-decisioner",   # ← the one that matters
        "agent_version": "2.3.1",
        "env": "prod",
        "owner": "risk-team",
    },
)

…or entirely from the environment:

export DOBBY_AGENT_KEY=kyc-decisioner
export DOBBY_AGENT_VERSION=2.3.1
export DOBBY_ENV=prod
export DOBBY_AGENT_OWNER=risk-team

agent_key is a name you choose and keep stable across deploys — not a secret, and not an ID Dobby issues you. It is what lets Dobby resolve every run of this agent, across replicas and restarts, to ONE canonical entry in your org-wide Agent Register.

	agent_key declared	not declared
Run's recorded identity_state	attested	unverifiable
Appears in the Agent Register	✅	❌
Counted in an evidence pack's Agent Identity & Coverage	✅	❌ — reported as unattributed

Dobby never infers a key for you. Identity is customer-declared by design: an inferred identity is an unprovable claim in an audit, so Dobby reports unverifiable rather than a false attestation. Runs without a key are still fully collected, governed and scanned — only the identity is unattested.

Changing an agent_key starts a new Register entry. Pick one per logical agent and keep it stable; use agent_version for releases of the same agent.

Lifecycle

from dobby_collector import init, shutdown

init(...)
# ... your agent runs ...
shutdown(timeout_seconds=5.0)  # Drains buffer + stops sender thread

shutdown() auto-fires via atexit if you forget, but call it explicitly when possible — atexit hooks have less time to drain before SIGTERM.

Status — v0.4.1 LIVE on PyPI · v0.4.2 pending publish

Shipped (Phases 1 → 6):

• ✅ Manual API: init / track / span / start_run / end_run / shutdown
• ✅ In-memory ring buffer + background sender + HTTP retries
• ✅ SQLite DLQ — events survive network outages + process crashes
• ✅ Field exclusion (exclude_fields=[...])
• ✅ LangChain auto-instrument — from dobby_collector.integrations.langchain import DobbyCallbackHandler
• ✅ CrewAI auto-instrument — from dobby_collector.integrations.crewai import DobbyCrewAIHandler (CrewAI ≥ 1.0)
• ✅ AutoGen auto-instrument — from dobby_collector.integrations.autogen import DobbyAutoGenLogHandler (AutoGen ≥ 0.4)
• ✅ OpenAI Assistants auto-instrument — from dobby_collector.integrations.openai_assistants import DobbyAssistantsHandler
• ✅ Google Gen AI (Gemini) auto-instrument — from dobby_collector.integrations.google_genai import instrument_google_genai (google-genai ≥ 1.0; patches client.models.generate_content + stream + async). Install: pip install dobby-collector[google_genai]
• ✅ call_gemini_json() — direct-REST Gemini call → parsed JSON, for agents that don't use the google-genai SDK — from dobby_collector.integrations.google_genai import call_gemini_json, LlmJsonError. Thinking disabled by default (a "thinking" model sharing maxOutputTokens between reasoning and the answer can truncate the JSON mid-response) + one bounded retry on a truncated/malformed response before raising typed LlmJsonError (never a bare JSONDecodeError). No extra install — pure httpx (already a core dependency).
• ✅ W3C Trace Context auto-emission per outbound batch (Option C, v0.4.0)

Quickstart pages (customer-facing on dobby-ai.com):

• /docs/sdk/python-collector/crewai — 5-minute CrewAI OSS instrumentation walkthrough
• /docs/sdk/python-collector/tracing — how W3C traceparent unlocks the Surrounding-mode Tracing Enabled governance control

Coming up (separate session):

• Node.js port — see .claude/plans/parallel-sessions/node-collector-sdk-port-2026-05-16.md

See CHANGELOG.md for the per-version detail.

License

MIT — see LICENSE.

Runnable examples

Six end-to-end working agents you can python immediately after pip install. All make REAL LLM calls (no mocks) — substitute the model/tool for whatever your stack uses:

• examples/langchain_real_agent.py — LangChain ReAct agent with DuckDuckGo search, instrumented via DobbyCallbackHandler. Install: pip install dobby-collector[langchain].
• examples/crewai_real_agent.py — CrewAI sequential crew (Researcher + Writer) with optional web search tool, instrumented via DobbyCrewAIHandler. Requires crewai>=1.0. Install: pip install dobby-collector[crewai].
• examples/autogen_real_agent.py — AutoGen AssistantAgent (gpt-4o-mini) with a Python tool function, instrumented via DobbyAutoGenLogHandler. Requires autogen-core>=0.4. Install: pip install dobby-collector[autogen].
• examples/openai_assistants_real_agent.py — OpenAI Assistants API with a function tool, streamed through client.beta.threads.runs.stream() with DobbyAssistantsHandler. Install: pip install dobby-collector[openai_assistants].
• examples/google_genai_real_agent.py — Gemini via the google-genai SDK (client.models.generate_content) plus a urllib tool call, auto-instrumented via instrument_google_genai() (monkey-patches the SDK — google-genai has no callback hook surface). Install: pip install dobby-collector[google_genai].
• examples/manual_api_real_agent.py — Plain Python + OpenAI SDK + urllib, instrumented via manual @track / span / start_run API. Recommended for custom orchestration / OpenAI Responses API / any framework without a Dobby auto-handler yet.

All scripts call https://dobby-ai.com and produce real workload_runs + compliance_scans rows visible in the UI within ~60s.

Customer walkthrough

Step-by-step onboarding guide (10–15 min, written for non-Dobby engineers):

• docs/customer-onboarding/python-sdk-walkthrough.md

Covers: wizard navigation → install → run → verify in UI + 7 troubleshooting scenarios (auth errors / DLQ / proxies / serverless / field exclusion).

Links

• Full spec: docs/spikes/dobby-collector-sdk-spec.md
• Dobby AI Platform: dobby-ai.com
• Documentation: docs.dobby-ai.com/sdk/python-collector