raindrop-deep-agents 0.0.1

Raindrop integration for Deep Agents

raindrop-deep-agents

Raindrop integration for Deep Agents (Python). Pass a single LangChain callback handler and Raindrop captures every LLM call, tool invocation, chain step, and agent action — with full OpenTelemetry trace nesting linked to the resulting dashboard event.

Installation

pip install raindrop-deep-agents deepagents langchain-core langchain-openai

Works with both wrapt < 2.0 and wrapt >= 2.0. The upstream opentelemetry-instrumentation-langchain package (latest 0.59.2) still calls wrapt.wrap_function_wrapper(module="...", ...) with the legacy module= keyword argument that wrapt 2.0 renamed to target=. Without a workaround the instrumentor crashes on activation and Traceloop silently swallows the error — traces never reach the dashboard while events keep working (tracked upstream at traceloop/openllmetry#4009 and Arize-ai/openinference#2996). On import, this package installs a small backwards-compat shim (raindrop_deep_agents/_wrapt_compat.py) that translates module= → target= for any caller using wrapt 2.x; on wrapt 1.x the shim is a no-op. We can drop the shim once the upstream instrumentors migrate.

Quick Start

from raindrop_deep_agents import RaindropDeepAgents
from deepagents import create_deep_agent
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

raindrop = RaindropDeepAgents(
    api_key="rk_...",
    user_id="user-123",
    convo_id="session-abc",
)

llm = ChatOpenAI(model="gpt-4o-mini")
agent = create_deep_agent(model=llm, system_prompt="Be concise.")

result = agent.invoke(
    {"messages": [HumanMessage(content="Hello!")]},
    config={"callbacks": [raindrop.handler]},
)

raindrop.shutdown()

Factory Function (alternative)

from raindrop_deep_agents import create_raindrop_deep_agents

raindrop = create_raindrop_deep_agents(api_key="rk_...", user_id="user-123")
agent = create_deep_agent(model=llm)
agent.invoke(
    {"messages": [HumanMessage(content="Hello")]},
    config={"callbacks": [raindrop.handler]},
)
raindrop.shutdown()

What Gets Captured

• LLM calls — model name, input, output, prompt + completion tokens, finish reason
• Tool calls — tool name, input arguments, output, duration as nested TOOL_CALL spans linked to the parent event (via interaction.track_tool())
• Chains — root chain input/output captured as the canonical event (the user's question → final assistant reply)
• Agent actions — tool selection events captured during the agent loop
• Errors — exception type and message captured for both LLM and tool failures
• Extended token categories — cached tokens (ai.usage.cached_tokens) and reasoning tokens (ai.usage.thoughts_tokens) when available from the provider (e.g. OpenAI, Anthropic)
• Finish reason — captured as ai.finish_reason in event properties ("stop", "length", "tool_calls", etc.)
• Tags & metadata — anything passed via LangChain config={"tags": [...], "metadata": {...}} is forwarded to event properties

Debug Mode

Enable verbose logging with debug=True:

raindrop = RaindropDeepAgents(
    api_key="rk_...",
    debug=True,
)

Identify Users

Associate events with a user after initialization:

raindrop.identify("user-123", {"name": "Alice", "plan": "pro"})

Track Signals

Send feedback, edits, or custom signals attached to a specific event:

# After an agent.invoke(...), attach a feedback signal to the event
# that just shipped.  ``raindrop.last_event_id`` is the canonical
# event_id the handler emitted for the root chain — using it (rather
# than a freshly generated UUID) ensures the signal links to the real
# dashboard row instead of creating an orphan signal.
if raindrop.last_event_id:
    raindrop.track_signal(
        event_id=raindrop.last_event_id,
        name="thumbs_up",
        signal_type="feedback",
        sentiment="POSITIVE",
        comment="user liked the answer",
    )

Flushing and Shutdown

raindrop.flush()     # flush pending events
raindrop.shutdown()  # flush + release resources (always call before process exit)

API Reference

RaindropDeepAgents

Parameter	Type	Default	Description
api_key	Optional[str]	None	Raindrop API key. If None, telemetry shipping is disabled (a UserWarning is issued).
user_id	Optional[str]	None	Stamped on every event. Falls back to "unknown" if not set.
convo_id	Optional[str]	None	Conversation/session ID — events sharing this value are grouped on the dashboard.
tracing_enabled	bool	True	Enable distributed tracing (activates the LangChain OTel instrumentor).
bypass_otel_for_tools	bool	True	Bypass OTEL for tool spans — emits OTLP directly to /v1/traces.
debug	bool	False	Enable DEBUG-level logging.

Methods

Method	Description
handler	Property — the LangChain callback handler. Pass into config={"callbacks": [...]} on agent.invoke(...).
last_event_id	Property — event_id of the most recently shipped root event (or None if none has landed yet). Use this to attach a track_signal to the agent invocation you just ran without having to poll the dashboard API.
identify(user_id, traits)	Identify a user with optional traits.
track_signal(event_id, name, signal_type, ...)	Track a feedback / edit / default signal against an event.
flush()	Flush all pending events to the Raindrop API.
shutdown()	Flush remaining events and release resources.

Async Support

The handler inherits from LangChain's BaseCallbackHandler and works with both synchronous and asynchronous Deep Agents invocations:

result = await agent.ainvoke(
    {"messages": [HumanMessage(content="Hello")]},
    config={"callbacks": [raindrop.handler]},
)

LangGraph Internals

Deep Agents is built on LangGraph. The handler keeps the outer LangGraph chain as the canonical root (so OTel association context is set before any span exists), and filters internal LangGraph nodes (__start__, __end__, ChannelWrite, ChannelRead, Branch:*, dunder names) so they don't pollute the dashboard.

Known Limitations

• Beta status — API surface may change in minor releases.
• Streaming — Token-by-token streaming events are not captured individually; only the final aggregated response per LLM call is tracked.
• Subagent isolation — Each subagent invoked via the task tool fires its own callback chain. Events are correctly grouped by convo_id, but the parent–child trace relationship across subagents is best-effort.
• OTel context propagation — The LangChain instrumentor sometimes does not propagate traceloop.association.properties.user_id baggage to every span produced inside a LangGraph callback context. The dashboard still shows the trace under the correct event because the SDK records trace_id on the root event's properties — but filtering the global Traces page by user_id may not surface every relevant span. Tracked upstream at traceloop/openllmetry#2271.
• Concurrent invocations on a shared handler — A single RaindropDeepAgents instance keeps one in-flight OTel association context (_interaction) at a time. Running multiple agent.invoke(...) / agent.ainvoke(...) calls concurrently with the same handler (e.g. await asyncio.gather(...) inside one process) will let later invocations share or overwrite the earlier one's interaction, scrambling the linkage between events and traces. Workaround: instantiate one RaindropDeepAgents per concurrent request (cheap — it's just a wrapper around module-level SDK state). Sequential invocations on a single instance are fine and exercised by the e2e test suite.
• Long chain inputs/outputs are truncated — Chain-level input and output captured on the root event are truncated to ~8 KB to stay within the SDK's payload limits. The full message history is still visible on the per-LLM child events; only the canonical-row summary is shortened.
• Python SDK feature surface — The Raindrop Python SDK uses module-level functions and does not expose the TypeScript-only EventShipper / TraceShipper low-level APIs. Equivalent capability is exposed as top-level methods (identify(), track_signal()) on this class.

Testing

cd packages/deep-agents-python
pip install -e ".[dev]"
python -m pytest tests/test_handler.py -v   # unit tests (no external services)

# E2E suite — exercises real Deep Agents + LLM and verifies dashboard ingestion.
# Tests skip in environments without keys (CI), so set them locally:
RAINDROP_WRITE_KEY=... \
OPENAI_API_KEY=... \
RAINDROP_DASHBOARD_TOKEN=... \
  python -m pytest tests/test_e2e.py -v

The dashboard token comes from app.raindrop.ai → DevTools → Network → any backend.raindrop.ai request → Authorization: Bearer ... (expires every ~30 minutes).

See examples/deep-agents-python-basic/ for a runnable end-to-end example with a tool call, identify(), and track_signal().