raindrop-openai-agents

Raindrop integration for the OpenAI Agents SDK (Python). Implements a TracingProcessor that automatically captures agent runs, LLM generations, tool calls, and handoffs and ships them to Raindrop.

Installation

pip install raindrop-openai-agents openai-agents

Quick Start

from raindrop_openai_agents import RaindropOpenAIAgents
from agents import Agent, Runner

raindrop = RaindropOpenAIAgents(
    api_key="your-write-key",
    user_id="user-123",
)
# Processor is auto-registered with the global trace provider

agent = Agent(name="Assistant", model="gpt-4o", instructions="Be helpful")
result = Runner.run_sync(agent, "Hello!")
print(result.final_output)

raindrop.flush()

Factory Function (Legacy)

The create_raindrop_openai_agents() factory is still available and now returns a RaindropOpenAIAgents instance:

from raindrop_openai_agents import create_raindrop_openai_agents

raindrop = create_raindrop_openai_agents(api_key="your-write-key", user_id="user-123")
raindrop.flush()

What Gets Captured

• Agent runs — trace-level events with workflow name
• LLM generations — model, input messages, output text, token usage
• Tool calls — individual tool spans with name, input, output, duration, and error tracking via the Interaction API
• Finish reason — extracted from response status or generation output (ai.finish_reason)
• Extended token categories — cached tokens (ai.usage.cached_tokens) and reasoning/thoughts tokens (ai.usage.thoughts_tokens) for OpenAI o1/o3 models
• Errors — error type and message captured in event properties (never interferes with agent execution)

Projects

Route events to a specific project by passing its slug as project_id:

raindrop = RaindropOpenAIAgents(
    api_key="your-write-key",
    project_id="support-prod",
)

project_id sets the X-Raindrop-Project-Id header on every event. Omit it (or pass "default") to use your org's default Production project, which is the existing behavior. The same option is accepted by the create_raindrop_openai_agents(...) factory. Invalid slugs are ignored with a warning and no header is sent.

API Reference

RaindropOpenAIAgents(api_key, user_id=None, convo_id=None, project_id=None, tracing_enabled=True, bypass_otel_for_tools=True, disable_auto_instrument=True, debug=False)

Library auto-instrumentation is opt-in

As of 0.0.4, disable_auto_instrument defaults to True: the integration no longer lets Traceloop monkey-patch every LLM client library it recognizes in your process (including the OpenAI client the Agents SDK itself drives). The tracing processor captures input/output, token usage, model name, tool calls, and handoffs directly from Agents SDK trace events, so no library patching is needed for full dashboards.

If you specifically want LLM-call-level spans from library instrumentation and have verified compatibility in your environment, opt back in with disable_auto_instrument=False.

Properties

Methods

Tool Call Tracking

Tool calls made by agents are automatically captured as individual tool spans. Each span includes:

• Name — the function/tool name
• Input — the arguments passed to the tool
• Output — the tool's return value
• Duration — execution time in milliseconds (computed from SDK span timestamps)
• Error — error message if the tool call failed

Tool spans are tracked via the Raindrop Interaction API (interaction.track_tool()), providing full visibility into agent tool usage alongside LLM generation data.

Extended Token Categories

For OpenAI o1/o3 models that report detailed token breakdowns, the integration captures:

These are reported alongside the standard ai.usage.prompt_tokens and ai.usage.completion_tokens.

Debug Mode

raindrop = RaindropOpenAIAgents(
    api_key="your-write-key",
    debug=True,  # enable verbose logging
)

Identify Users

raindrop.identify("user-42", traits={"plan": "pro", "company": "Acme"})

Track Signals

raindrop.track_signal(
    event_id="evt_abc123",
    name="thumbs_up",
    signal_type="feedback",
    sentiment="POSITIVE",
    comment="Great answer!",
)

Flush & Shutdown

raindrop.flush()     # flush pending data
raindrop.shutdown()  # flush + release resources

Tool catalog capture (ai.prompt.tools)

This wrapper is override-only. The Agents SDK trace API it builds on does not show the processor the tool list a model call received: AgentSpanData.tools is a list of tool names with no schema, GenerationSpanData carries no tools, and ResponseSpanData.response.tools (the full list) is only filled in when the span ends, after the model span has already started. Inferring a list from any of those would either invent schemas or overstate what the call received, so nothing is inferred and the attribute stays absent unless you pass tools=.

Model spans come from the OpenLLMetry OpenAI instrumentor, which only runs with disable_auto_instrument=False. OpenLLMetry 0.62 writes the request's tools to gen_ai.tool.definitions, not llm.request.functions.*, so raindrop-ai does not rebuild ai.prompt.tools from the instrumentor either. With the default disable_auto_instrument=True there is no model span at all, and nothing to carry the attribute.

Override:

# Default for every run this wrapper traces (goes through begin(..., tools=...))
rd = RaindropOpenAIAgents(api_key="...", user_id="u1", disable_auto_instrument=False,
                          tools=[{"type": "function", "name": "get_weather", "inputSchema": {...}}])

# Per run, through the Agents SDK's own per-run metadata channel
from agents import RunConfig
Runner.run_sync(agent, "weather?", run_config=RunConfig(trace_metadata={"raindrop_tools": [...]}))

The per-run value replaces the constructor default; tools=[] at either level records an empty catalog. Hosted tools (web search, file search, ...) are recorded as {"type": "provider-defined", "name": ...} when you list them that way; the wrapper never invents a schema for them.

Semantics shared with every Raindrop SDK:

• Attribute absent: the wrapper could not see the tool list (or there is no model span, see below).
• []: the model had no tools.
• Override replaces, never merges. A tools= you pass wins over anything inferred; tools=[] records an empty catalog.
• Content gate. Tool definitions are prompt content. With TRACELOOP_TRACE_CONTENT=false nothing is recorded, overrides included.
• Malformed entries are skipped with a debug log; nothing here raises into your application.

Accepted shapes for tools=: canonical {"type": "function", "name", "description", "inputSchema"}, OpenAI {"type": "function", "function": {...}}, Anthropic {"name", "description", "input_schema"}, and {"type": "provider-defined", "name", ...}. The core normalises them; the JSON Schema is kept exactly as given.

Needs raindrop-ai>=0.0.70 (begin(..., tools=...)); on an older core the wrapper logs at debug and records nothing.

Known Limitations

• Multi-response traces — in multi-agent workflows, only the last response's data survives per trace.

Full Documentation

docs.raindrop.ai/integrations/openai-agents

Application Git metadata

RaindropOpenAIAgents(...) and create_raindrop_openai_agents(...) accept the keyword-only app_git option. It defaults to True: explicit Raindrop Git environment or deployment context is applied immediately, and the base SDK may perform one bounded background local-Git lookup from the process working directory. Event capture, flush, and shutdown never wait for that lookup. Pass False to disable enrichment, or pass an AppGitOptions mapping with commit_sha, commit_dirty, branch, source_directory, detect_branch, and/or auto_detect. Automatic branch discovery remains opt-in through detect_branch=True (or RAINDROP_GIT_DETECT_BRANCH=true).

For an ordinary in-process application, the process working directory is treated as the application-under-test checkout. A remote, coding, workflow, or observer process must not rely on its own checkout: pass app_git=False, provide explicit revision values, or set source_directory to the actual application checkout. Canonical per-operation properties remain authoritative. When supplying client=, configure app_git while constructing that Raindrop client; the supplied client is authoritative and the wrapper's app_git argument does not reconfigure it.

Release order is deliberate: first publish the base SDK feature, then publish the wrapper feature release with its minimum dependency coordinated to that base release. The existing raindrop-ai lower bound remains compatible, but application Git metadata is unavailable on an older core and must not be claimed complete until the base is upgraded. Until coordination assigns a released version, the wrapper checks for an explicit base app_git parameter and omits the option when unsupported. Explicit non-default configuration is debug-logged and omitted. Unsupported app_git is determined by signature inspection before construction, not by retrying initialization after a TypeError; Git configuration adds no initialization attempts and does not change any existing framework-specific initialization fallback.

Testing

cd packages/openai-agents-python
pip install -e ".[dev]"
python -m pytest tests/ -v   # unit tests (no external services)

End-to-end behavior is verified by the cross-SDK conformance harness. This package ships a thin conformance driver at conformance/driver.py that maps the shared scenario corpus onto the wrapper's public API; known gaps are tracked as ticket-linked entries in conformance/failures.txt. The fault lane runs on every PR touching packages/*-python/** (.github/workflows/conformance-wrappers-python.yml) against a local capture server; the prod lane verifies delivery by reading back through the public Query API. The harness is pinned by commit SHA (HARNESS_REF). See the harness docs: HOW-IT-WORKS · AGENTS · README.

License

MIT