raindrop-azure-openai

Raindrop observability integration for Azure OpenAI (Python). Automatically captures chat.completions.create() calls by wrapping AzureOpenAI and AsyncAzureOpenAI clients.

Installation

pip install raindrop-azure-openai openai

Quick Start

from raindrop_azure_openai import RaindropAzureOpenAI
from openai import AzureOpenAI

raindrop = RaindropAzureOpenAI(
    api_key="rk_...",
    user_id="user-123",
    debug=False,  # set True for verbose logging
)

client = AzureOpenAI(
    azure_endpoint="https://your-resource.openai.azure.com",
    api_key="...",
    api_version="2024-10-21",
)
wrapped = raindrop.wrap(client)

response = wrapped.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
)

raindrop.shutdown()

Projects

Route events to a specific project by passing its slug as project_id:

raindrop = RaindropAzureOpenAI(
    api_key="rk_...",
    project_id="support-prod",
)

project_id sets the X-Raindrop-Project-Id header on every event. Omit it (or pass "default") to use your org's default Production project, which is the existing behavior. The same option is accepted by the create_raindrop_azure_openai(...) factory. Invalid slugs are ignored with a warning and no header is sent.

Debug Mode

raindrop = RaindropAzureOpenAI(api_key="rk_...", debug=True)

When debug=True, verbose logs are emitted to the raindrop_azure_openai logger at DEBUG level.

Async Support

from raindrop_azure_openai import RaindropAzureOpenAI
from openai import AsyncAzureOpenAI

raindrop = RaindropAzureOpenAI(api_key="rk_...", user_id="user-123")

client = AsyncAzureOpenAI(
    azure_endpoint="https://your-resource.openai.azure.com",
    api_key="...",
    api_version="2024-10-21",
)
wrapped = raindrop.wrap(client)

response = await wrapped.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
)

raindrop.shutdown()

User Identification

raindrop.identify("user-123", traits={"plan": "pro", "name": "Alice"})

Tracking Signals

raindrop.track_signal(
    event_id="evt-abc",
    name="thumbs_up",
    signal_type="feedback",
    sentiment="POSITIVE",
    comment="Great response!",
)

Flushing and Shutdown

raindrop.flush()     # flush pending data
raindrop.shutdown()  # flush + release resources

Factory Function (Backwards-Compatible)

from raindrop_azure_openai import create_raindrop_azure_openai

raindrop = create_raindrop_azure_openai(api_key="rk_...", user_id="user-123")
# raindrop is now a RaindropAzureOpenAI instance with .wrap(), .flush(), .shutdown()

What Gets Captured

• Chat completions — input messages, output text, model, token usage
• Finish reason — azure_openai.finish_reason (stop, length, content_filter, tool_calls)
• Extended tokens — ai.usage.cached_tokens (prompt cache hits) and ai.usage.thoughts_tokens (reasoning tokens for o1/o3 models)
• Errors — error type and message captured as properties, then re-raised to the caller
• Async support — both sync (AzureOpenAI) and async (AsyncAzureOpenAI) clients are instrumented

Configuration

Library auto-instrumentation is opt-in

As of 0.0.4, disable_auto_instrument defaults to True: the integration no longer lets Traceloop monkey-patch every LLM client library it recognizes in your process (the OpenAI client itself, Anthropic, botocore, etc.). The wrapper captures input/output, token usage, model name, and finish_reason directly from the wrapped client's responses, so no library patching is needed for full dashboards.

If you specifically want LLM-call-level spans from library instrumentation and have verified compatibility in your environment, opt back in with disable_auto_instrument=False.

Tool catalog capture (ai.prompt.tools)

Every model span records the tools the request carried as ai.prompt.tools: an OTel string array, one JSON document per tool with name, description and the full JSON Schema as inputSchema ({"type": "function", ...}), or {"type": "provider-defined", ...} for provider-owned tools. The capture is exact, per call: the wrapped chat.completions.create(...) sees the request's own tools= (or legacy functions=) and binds that list through the core's prompt_tools_scope mechanism for the duration of the call, sync and async. A request with neither records []. Streaming (stream=True) is covered because the instrumentor starts the model span inside create(), while the list is bound; the response being consumed later does not matter.

Model spans come from the OpenLLMetry OpenAI instrumentor, which only runs with disable_auto_instrument=False. OpenLLMetry 0.62 writes the same list to gen_ai.tool.definitions, not llm.request.functions.*, so raindrop-ai does not rebuild ai.prompt.tools from the instrumentor; the wrapper-side scope is what puts the attribute on the span. With the default disable_auto_instrument=True there is no model span at all, and nothing to carry the attribute.

Override:

rd = RaindropAzureOpenAI(api_key="...", user_id="u1", disable_auto_instrument=False,
                         tools=[{"type": "function", "name": "get_weather", "inputSchema": {...}}])
client = rd.wrap(AzureOpenAI(...))                 # constructor default applies to every call
client = rd.wrap(AzureOpenAI(...), tools=[...])    # per-wrap value replaces the default

tools=[] at either level records an empty catalog. An enclosing raindrop.prompt_tools(...) / begin(..., tools=...) scope is also treated as an override and is never shadowed by the request's list.

Semantics shared with every Raindrop SDK:

• Attribute absent: the wrapper could not see the tool list (or there is no model span, see below).
• []: the model had no tools.
• Override replaces, never merges. A tools= you pass wins over anything inferred; tools=[] records an empty catalog.
• Content gate. Tool definitions are prompt content. With TRACELOOP_TRACE_CONTENT=false nothing is recorded, overrides included.
• Malformed entries are skipped with a debug log; nothing here raises into your application.

Accepted shapes for tools=: canonical {"type": "function", "name", "description", "inputSchema"}, OpenAI {"type": "function", "function": {...}}, Anthropic {"name", "description", "input_schema"}, and {"type": "provider-defined", "name", ...}. The core normalises them; the JSON Schema is kept exactly as given.

Needs raindrop-ai>=0.0.70; on an older core the wrapper logs at debug and records nothing.

Double-Wrap Protection

Calling wrap() on an already-instrumented client is a safe no-op — the client is returned unchanged.

Full Documentation

See the Raindrop docs for the complete reference.

Application Git metadata

RaindropAzureOpenAI(...) and create_raindrop_azure_openai(...) accept the keyword-only app_git option. It defaults to True: explicit Raindrop Git environment or deployment context is applied immediately, and the base SDK may perform one bounded background local-Git lookup from the process working directory. Event capture, flush, and shutdown never wait for that lookup. Pass False to disable enrichment, or pass an AppGitOptions mapping with commit_sha, commit_dirty, branch, source_directory, detect_branch, and/or auto_detect. Automatic branch discovery remains opt-in through detect_branch=True (or RAINDROP_GIT_DETECT_BRANCH=true).

For an ordinary in-process application, the process working directory is treated as the application-under-test checkout. A remote, coding, workflow, or observer process must not rely on its own checkout: pass app_git=False, provide explicit revision values, or set source_directory to the actual application checkout. Canonical per-operation properties remain authoritative. When supplying client=, configure app_git while constructing that Raindrop client; the supplied client is authoritative and the wrapper's app_git argument does not reconfigure it.

Release order is deliberate: first publish the base SDK feature, then publish the wrapper feature release with its minimum dependency coordinated to that base release. The existing raindrop-ai lower bound remains compatible, but application Git metadata is unavailable on an older core and must not be claimed complete until the base is upgraded. Until coordination assigns a released version, the wrapper checks for an explicit base app_git parameter and omits the option when unsupported. Explicit non-default configuration is debug-logged and omitted. Unsupported app_git is determined by signature inspection before construction, not by retrying initialization after a TypeError; Git configuration adds no initialization attempts and does not change any existing framework-specific initialization fallback.

Testing

cd packages/azure-openai-python
pip install -e ".[dev]"
python -m pytest tests/ -v   # unit tests (no external services)

End-to-end behavior is verified by the cross-SDK conformance harness. This package ships a thin conformance driver at conformance/driver.py that maps the shared scenario corpus onto the wrapper's public API; known gaps are tracked as ticket-linked entries in conformance/failures.txt. The fault lane runs on every PR touching packages/*-python/** (.github/workflows/conformance-wrappers-python.yml) against a local capture server; the prod lane verifies delivery by reading back through the public Query API. The harness is pinned by commit SHA (HARNESS_REF). See the harness docs: HOW-IT-WORKS · AGENTS · README.

License

MIT