raindrop-deep-agents

Raindrop integration for Deep Agents (Python). Pass a single LangChain callback handler and Raindrop captures every LLM call, tool invocation, chain step, and agent action — with full OpenTelemetry trace nesting linked to the resulting dashboard event.

Installation

pip install raindrop-deep-agents deepagents langchain-core langchain-openai

Works with both wrapt < 2.0 and wrapt >= 2.0. The upstream opentelemetry-instrumentation-langchain package (latest 0.59.2) still calls wrapt.wrap_function_wrapper(module="...", ...) with the legacy module= keyword argument that wrapt 2.0 renamed to target=. Without a workaround the instrumentor crashes on activation and Traceloop silently swallows the error — traces never reach the dashboard while events keep working (tracked upstream at traceloop/openllmetry#4009 and Arize-ai/openinference#2996). On import, this package installs a small backwards-compat shim (raindrop_deep_agents/_wrapt_compat.py) that translates module= → target= for any caller using wrapt 2.x; on wrapt 1.x the shim is a no-op. We can drop the shim once the upstream instrumentors migrate.

Quick Start

from raindrop_deep_agents import RaindropDeepAgents
from deepagents import create_deep_agent
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

raindrop = RaindropDeepAgents(
    api_key="rk_...",
    user_id="user-123",
    convo_id="session-abc",
)

llm = ChatOpenAI(model="gpt-4o-mini")
agent = create_deep_agent(model=llm, system_prompt="Be concise.")

result = agent.invoke(
    {"messages": [HumanMessage(content="Hello!")]},
    config={"callbacks": [raindrop.handler]},
)

raindrop.shutdown()

Factory Function (alternative)

from raindrop_deep_agents import create_raindrop_deep_agents

raindrop = create_raindrop_deep_agents(api_key="rk_...", user_id="user-123")
agent = create_deep_agent(model=llm)
agent.invoke(
    {"messages": [HumanMessage(content="Hello")]},
    config={"callbacks": [raindrop.handler]},
)
raindrop.shutdown()

What Gets Captured

• LLM calls — model name, input, output, prompt + completion tokens, finish reason
• Tool calls — tool name, input arguments, output, duration as nested TOOL_CALL spans linked to the parent event (via interaction.track_tool())
• Chains — root chain input/output captured as the canonical event (the user's question → final assistant reply)
• Agent actions — tool selection events captured during the agent loop
• Errors — exception type and message captured for both LLM and tool failures
• Extended token categories — cached tokens (ai.usage.cached_tokens) and reasoning tokens (ai.usage.thoughts_tokens) when available from the provider (e.g. OpenAI, Anthropic)
• Finish reason — captured as ai.finish_reason in event properties ("stop", "length", "tool_calls", etc.)
• Tags & metadata — anything passed via LangChain config={"tags": [...], "metadata": {...}} is forwarded to event properties

Projects

Route events to a specific project by passing its slug as project_id:

raindrop = RaindropDeepAgents(
    api_key="rk_...",
    project_id="support-prod",
)

project_id sets the X-Raindrop-Project-Id header on every event. Omit it (or pass "default") to use your org's default Production project, which is the existing behavior. The same option is accepted by the create_raindrop_deep_agents(...) factory. Invalid slugs are ignored with a warning and no header is sent.

Debug Mode

Enable verbose logging with debug=True:

raindrop = RaindropDeepAgents(
    api_key="rk_...",
    debug=True,
)

Identify Users

Associate events with a user after initialization:

raindrop.identify("user-123", {"name": "Alice", "plan": "pro"})

Track Signals

Send feedback, edits, or custom signals attached to a specific event:

# After an agent.invoke(...), attach a feedback signal to the event
# that just shipped.  ``raindrop.last_event_id`` is the canonical
# event_id the handler emitted for the root chain — using it (rather
# than a freshly generated UUID) ensures the signal links to the real
# dashboard row instead of creating an orphan signal.
if raindrop.last_event_id:
    raindrop.track_signal(
        event_id=raindrop.last_event_id,
        name="thumbs_up",
        signal_type="feedback",
        sentiment="POSITIVE",
        comment="user liked the answer",
    )

Flushing and Shutdown

raindrop.flush()     # flush pending events
raindrop.shutdown()  # flush + release resources (always call before process exit)

API Reference

RaindropDeepAgents

Methods

Async Support

The handler inherits from LangChain's BaseCallbackHandler and works with both synchronous and asynchronous Deep Agents invocations:

result = await agent.ainvoke(
    {"messages": [HumanMessage(content="Hello")]},
    config={"callbacks": [raindrop.handler]},
)

LangGraph Internals

Deep Agents is built on LangGraph. The handler keeps the outer LangGraph chain as the canonical root (so OTel association context is set before any span exists), and filters internal LangGraph nodes (__start__, __end__, ChannelWrite, ChannelRead, Branch:*, dunder names) so they don't pollute the dashboard.

Known Limitations

• Beta status — API surface may change in minor releases.
• Streaming — Token-by-token streaming events are not captured individually; only the final aggregated response per LLM call is tracked.
• Subagent isolation — Each subagent invoked via the task tool fires its own callback chain. Events are correctly grouped by convo_id, but the parent–child trace relationship across subagents is best-effort.
• OTel context propagation — The LangChain instrumentor sometimes does not propagate traceloop.association.properties.user_id baggage to every span produced inside a LangGraph callback context. The dashboard still shows the trace under the correct event because the SDK records trace_id on the root event's properties — but filtering the global Traces page by user_id may not surface every relevant span. Tracked upstream at traceloop/openllmetry#2271.
• Concurrent invocations on a shared handler — A single RaindropDeepAgents instance keeps one in-flight OTel association context (_interaction) at a time. Running multiple agent.invoke(...) / agent.ainvoke(...) calls concurrently with the same handler (e.g. await asyncio.gather(...) inside one process) will let later invocations share or overwrite the earlier one's interaction, scrambling the linkage between events and traces. Workaround: instantiate one RaindropDeepAgents per concurrent request (cheap — it's just a wrapper around module-level SDK state). Sequential invocations on a single instance are fine and exercised by the e2e test suite.
• Long chain inputs/outputs are truncated — Chain-level input and output captured on the root event are truncated to ~8 KB to stay within the SDK's payload limits. The full message history is still visible on the per-LLM child events; only the canonical-row summary is shortened. As of 0.0.2 the truncation is also enforced during serialization (cost proportional to the cap, not the payload — a multi-MB history or multi-modal content list can't stall the event loop inside a synchronous callback), with structured payloads marked ...[truncated by raindrop].
• Python SDK feature surface — The Raindrop Python SDK uses module-level functions and does not expose the TypeScript-only EventShipper / TraceShipper low-level APIs. Equivalent capability is exposed as top-level methods (identify(), track_signal()) on this class.

Application Git metadata

RaindropDeepAgents(...) and create_raindrop_deep_agents(...) accept the keyword-only app_git option. It defaults to True: explicit Raindrop Git environment or deployment context is applied immediately, and the base SDK may perform one bounded background local-Git lookup from the process working directory. Event capture, flush, and shutdown never wait for that lookup. Pass False to disable enrichment, or pass an AppGitOptions mapping with commit_sha, commit_dirty, branch, source_directory, detect_branch, and/or auto_detect. Automatic branch discovery remains opt-in through detect_branch=True (or RAINDROP_GIT_DETECT_BRANCH=true).

For an ordinary in-process application, the process working directory is treated as the application-under-test checkout. A remote, coding, workflow, or observer process must not rely on its own checkout: pass app_git=False, provide explicit revision values, or set source_directory to the actual application checkout. Canonical per-operation properties remain authoritative. When supplying client=, configure app_git while constructing that Raindrop client; the supplied client is authoritative and the wrapper's app_git argument does not reconfigure it.

Release order is deliberate: first publish the base SDK feature, then publish the wrapper feature release with its minimum dependency coordinated to that base release. The existing raindrop-ai lower bound remains compatible, but application Git metadata is unavailable on an older core and must not be claimed complete until the base is upgraded. Until coordination assigns a released version, the wrapper checks for an explicit base app_git parameter and omits the option when unsupported. Explicit non-default configuration is debug-logged and omitted. Unsupported app_git is determined by signature inspection before construction, not by retrying initialization after a TypeError; Git configuration adds no initialization attempts and does not change any existing framework-specific initialization fallback.

Testing

cd packages/deep-agents-python
pip install -e ".[dev]"
python -m pytest tests/test_handler.py -v   # unit tests (no external services)

End-to-end behavior is verified by the cross-SDK conformance harness. This package ships a thin conformance driver at conformance/driver.py that maps the shared scenario corpus onto the wrapper's public API; known gaps are tracked as ticket-linked entries in conformance/failures.txt. The fault lane runs on every PR touching packages/*-python/** (.github/workflows/conformance-wrappers-python.yml) against a local capture server; the prod lane verifies delivery by reading back through the public Query API. The harness is pinned by commit SHA (HARNESS_REF). See the harness docs: HOW-IT-WORKS · AGENTS · README.

See examples/deep-agents-python-basic/ for a runnable end-to-end example with a tool call, identify(), and track_signal().