struct-sdk

OpenTelemetry instrumentation for AI agents in Python. Captures spans, token usage, and message content from the Anthropic SDK, the OpenAI SDK (Responses API), google-genai (Gemini), LiteLLM, pydantic-ai, the OpenAI Agents SDK, MCP clients, the Claude Agent SDK, LangChain / LangGraph, LlamaIndex, CrewAI, Microsoft AutoGen, and AG2, and exports them to struct.ai for observability.

A TypeScript version is available as @struct-ai/sdk. Both SDKs produce structurally identical traces, so a single agent system can mix languages without a fragmented view.

Install

pip install struct-sdk
# optional — the SDK auto-instruments these if present
pip install anthropic
pip install openai
pip install pydantic-ai-slim
pip install openai-agents
pip install mcp
pip install google-genai
pip install "litellm[mcp]"   # LiteLLM needs mcp for calls with tools
pip install claude-agent-sdk
pip install langchain-core langgraph
pip install llama-index-core llama-index-llms-openai   # or the provider integration you use

Requires Python 3.10+.

Quickstart

Get an ingest key from app.struct.ai/settings?tab=ingest-keys, then call struct.init() once at startup and wrap your agent loop:

import os
from struct_sdk import struct

struct.init(
    ingest_key=os.environ["STRUCT_INGEST_KEY"],  # or pass the string directly
    service_name="my-agent",
    environment="production",
)

import anthropic
client = anthropic.AsyncAnthropic()

# Decorate each tool — auto-captures arguments + result + tool_call_id.
@struct.tool()
async def search(query: str):
    ...

async with struct.agent(name="checkout"):
    msg = await client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=1024,
        messages=[{"role": "user", "content": "plan my checkout flow"}],
    )
    result = await search(query="...")

What gets traced

Framework integration

struct.init() takes the same parameters regardless of which framework you're instrumenting. Required: ingest_key (get one at app.struct.ai/settings?tab=ingest-keys). Recommended: service_name, environment.

What you need to do beyond init() depends on whether you're using an agent framework (which has built-in concepts of agents and tools) or an LLM SDK directly (which only knows about chat completions). The SDK auto-instruments both, but only agent frameworks get full agent + tool spans for free — when you call an LLM SDK directly, you have to tell the SDK where the agent and tool boundaries are.

Call init() once, as early as possible, before the instrumented libraries are imported.

Agent frameworks — fully auto-instrumented

For these, calling struct.init() is the only setup. Agent, tool, chat, and retrieval spans all emit automatically.

Claude Agent SDK

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="claude-agent")

from claude_agent_sdk import ClaudeAgentOptions, query
# Telemetry is generated by Claude Code itself in the subprocess and
# exported directly via OTLP. struct.init() configures the ingest
# credentials on ClaudeAgentOptions; subagents inherit them automatically.

pydantic-ai

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="support-agent")

from pydantic_ai import Agent
agent = Agent("openai:gpt-5.5", name="support")
result = await agent.run("Refund order 42", conversation_id="thread-42")

pydantic-ai already emits OpenTelemetry GenAI spans; struct.init() turns that on for every agent (Agent.instrument_all) with Struct's tracer provider, unless you already set your own default (for example with logfire). You get pydantic-ai's invoke_agent, chat and execute_tool spans, with one chat span per model call (Struct's OpenAI and Anthropic patches stay out of the way for pydantic-ai's own provider calls; model calls your code makes while reading a stream keep their spans). During agent runs, Struct guards telemetry faults and caps content. Realtime sessions and direct InstrumentedModel exception recording are outside this guarantee.

• Content: in event modes, each chat call's messages go to one operation.details event. Span modes keep chat content on spans. Tool arguments and results remain on execute_tool spans in every mode but none.
• Sessions: on Struct's provider, a struct.agent(session_id=...) around the run wins; otherwise pydantic-ai's conversation_id is used. Instrumentation configured on your own provider keeps its conversation IDs.
• Turn it off with struct.init(..., instrumentation_options={"pydantic_ai": {"enabled": False}}), or for one agent with agent.instrument = False. An Agent.instrument_all(False) before struct.init() is pydantic-ai's initial value, so it does not keep pydantic-ai off.

OpenAI Agents SDK

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="support-agents")

from agents import Agent, Runner, RunConfig
result = await Runner.run(triage_agent, "Refund order 42",
                          run_config=RunConfig(group_id=conversation_id))

struct.init() adds a tracing processor to the Agents SDK (add_trace_processor), so OpenAI's own trace upload keeps working. Each run is one trace:

invoke_agent Agent workflow          # the run (RunConfig.workflow_name)
├── invoke_agent Triage              # each agent that ran
│   ├── chat gpt-5.4-mini            # each model call, with usage
│   ├── execute_tool get_weather     # each function tool, with its call id
│   └── execute_tool transfer_to_billing   # a handoff is a tool call
└── invoke_agent Billing
    └── chat gpt-5.4-mini

• Sessions. RunConfig(group_id=...) becomes gen_ai.conversation.id. Wrapping the run in struct.agent(session_id=...) makes that span the run's root instead (no separate invoke_agent Agent workflow) and its session wins.

• One span per model call. The processor owns chat spans, including streamed runs; the OpenAI patch makes no span of its own while it does, only recording the requested model, which the Agents SDK's Responses spans do not carry. Non-streamed Chat Completions and LiteLLM model spans have no response id or model (the Agents SDK records only the message).

• Nesting. While a tool or model call runs, its span is the current OpenTelemetry span, so your own spans inside a tool (and HTTP client spans in full-app mode) nest under it.

• Content needs both switches on: the Agents SDK's trace_include_sensitive_data (on by default) and a Struct content_capture mode other than none. Usage is always recorded.

• Turning Agents tracing off (OPENAI_AGENTS_DISABLE_TRACING=1, RunConfig(tracing_disabled=True)) also removes Struct's agent and tool spans; only the OpenAI patch's chat spans for non-streamed calls remain. To stop only the upload to OpenAI, replace the processor list and keep Struct's:

  from agents import set_trace_processors
  from struct_sdk.openai_agents import get_tracing_processor
  set_trace_processors([get_tracing_processor()])  # after struct.init()
  

Gemini and Google ADK

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="planner")

from google import genai
client = genai.Client()  # or genai.Client(vertexai=True, project=..., location=...)
response = client.models.generate_content(model="gemini-3.1-flash-lite", contents="Plan my day")

Each generate_content call (sync or client.aio) is a generate_content {model} span. Wrap your loop in struct.agent() and your tools in struct.tool() as for the other LLM SDKs.

Google ADK traces itself (agents, tools and model calls) through the global OpenTelemetry provider, which Struct owns or attaches to in the default full-app mode, so ADK runs need no extra setup; Struct adds no second span for a model call ADK already traces. With auto_instrument=False and no host OpenTelemetry, Struct sees ADK's model calls but not its agent and tool spans. ADK puts message content in its spans by default; set ADK's own ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS=false to stop that.

If you install opentelemetry-instrumentation-google-genai yourself, Struct leaves google-genai to it. Default full-app mode also leaves Datadog-owned google-genai bindings alone; explicit AI-only mode retains Struct’s isolated instrumentation.

LlamaIndex

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="my-agent")

from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

llm = OpenAI(
    model="gpt-4.1-mini",
    # Streamed calls report token usage only when asked to.
    additional_kwargs={"stream_options": {"include_usage": True}},
)
agent = FunctionAgent(name="weather", tools=[...], llm=llm)
result = await agent.run(user_msg="Weather in Paris?")

Each run is an invoke_agent span with its model calls (chat), tool calls (execute_tool) and retrievals (retrieval) under it, even though LlamaIndex runs workflow steps on separate tasks. Wrap the run in struct.agent(session_id=...) to group runs into a conversation. Completions, embeddings and query engines are not spans of their own; their LLM and retriever calls are.

CrewAI

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="my-crew")

from crewai import Agent, Crew, Task
crew = Crew(name="weather-crew", agents=[...], tasks=[...])
with struct.agent(name="support", session_id=conversation_id):
    crew.kickoff()

The crew and each agent execution are invoke_agent spans, with the agent's model calls (chat) and tool calls (execute_tool) under it, across the threads CrewAI runs agents and async tasks on. Agent spans are named by the role as configured; the resolved goal (with kickoff inputs) is recorded only when content is captured. A cancelled akickoff() (or an interrupted kickoff()) ends the run's spans as errors. Requires CrewAI 1.15.13 or later (older releases are left alone); there is no struct-sdk[crewai] extra. Install CrewAI separately as you already do; the SDK activates the integration when it is present.

AutoGen

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="my-agent")  # full-app mode, the default

from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

agent = AssistantAgent(name="weather", model_client=OpenAIChatCompletionClient(model="gpt-4.1-mini"), tools=[...])
async with struct.agent(name="support", session_id=conversation_id):
    await agent.run(task="Weather in Paris?")

AutoGen (0.7+) traces its agents and tools through the global OpenTelemetry provider, which Struct owns or attaches to in the default full-app mode; with auto_instrument=False only Struct's chat spans arrive. The struct.agent() session reaches AutoGen's own spans only when Struct owns the provider (owned full-app mode); attached to your provider, only Struct's chat spans carry it. Streamed model calls (model_client_stream=True) get no AutoGen span; the openai integration traces them. This is Microsoft AutoGen (autogen-agentchat), not AG2.

AG2

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="my-agents")

from autogen import ConversableAgent
with struct.agent(name="support", session_id=conversation_id):
    user.initiate_chat(assistant, message="Weather in Paris?")

AG2 0.x (pip install "ag2[openai]<1", imported as autogen) gets a chat span per model call and an execute_tool span per tool call; wrap the conversation in struct.agent() for an agent span and a session. The chat span describes the request AG2 actually sent (the client that answered, its merged config, instantiated templates), for streamed calls and the V2 clients too; Anthropic and OpenAI (Responses and Chat Completions) calls that are not streamed keep the provider's cache counts. Async chats (a_initiate_chat) run model calls on an executor thread. OTel thread instrumentation carries trace parentage; the wrapper can recover the session from a carried current Struct span. Owned full-app mode enables that instrumentor when installed; a host can also supply it. Without thread-context propagation, model spans start new traces without the session. AG2's own OpenTelemetry (autogen.opentelemetry in 0.x, TelemetryMiddleware in 1.x) also works: pass it trace.get_tracer_provider() (Struct's in owned mode, yours when Struct attaches to it) and Struct stamps the session on its spans and adds no duplicates under them.

LangChain / LangGraph (with an agent or graph)

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="my-graph")

from langgraph.prebuilt import create_react_agent
# Pregel / CompiledStateGraph / AgentExecutor invocations get invoke_agent
# spans. BaseChatModel calls get chat spans. BaseTool.invoke gets
# execute_tool spans. BaseRetriever.invoke gets retrieval spans.

• Sub-graphs and create_agent sub-agents nest under their parent graph; configurable.thread_id is the session, so a run resumed after an interrupt() stays in the same session.
• interrupt() and Command(graph=Command.PARENT) are control flow: the tool or graph span is not marked as an error.
• While a tool runs, its execute_tool span is the current span: your own spans inside the tool, and full-app database/HTTP spans, nest under it. Cancellation restores the caller’s context even when LangChain skips its end/error callbacks. Plain function nodes get no Struct span; their work inherits existing host or struct.agent() context. Automatic graph spans end on interrupt; resume starts a separate trace only without an enclosing span.

Recommended pattern: wrap LangChain entry points in struct.agent

For multi-turn HTTP-style usage (every request continues the same conversation), wrap your request handler in struct.agent(session_id=conversation_id). This is the struct-native replacement for with ls.tracing_context(parent=run_tree): and gives you two things you can't get from configurable.thread_id alone:

1. Threading without per-call config plumbing. Every nested LangChain call inherits the conversation id via the SDK's ambient contextvar — you don't have to ensure each compiled_graph.ainvoke gets thread_id on its config.
2. One trace per request. struct.agent creates a parent OTel span so all the LangChain work for the request nests under one trace (clean tree, "Subagents" / "Spawned by" UI links work). Without it, each .invoke() becomes its own root trace, and the UI's session list shows a non-deterministic agent name (omni_agent, LangGraph, the first sub-agent it sees…).

Migrating from LangSmith:

# Before — LangSmith convention, fragments under struct-sdk
with ls.tracing_context(parent=run_tree):
    await orchestrator.ainvoke(inputs, config=config)

# After — struct-native, threads correctly, no langsmith dep
async with struct.agent(session_id=conversation_id):
    await orchestrator.ainvoke(inputs, config=config)

Also works as a sync context manager (with struct.agent(...)) for non-async handlers.

LLM SDKs used directly — manual agent + tool scopes required

When you call an LLM SDK directly (no agent framework wrapping it), only chat spans emit automatically. You need to wrap your agent loop in struct.agent() and each tool execution in struct.tool() so the SDK knows where to put the agent and tool boundaries — otherwise you'll see free-floating chat spans with no agent or tool context around them.

Anthropic SDK (raw)

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="checkout-agent")

import anthropic
client = anthropic.AsyncAnthropic()

# Recommended: define each tool as a function and DECORATE it. The decorator
# auto-captures the tool's arguments + result on the execute_tool span and
# auto-fills tool_call_id from the preceding Anthropic response — no manual
# bookkeeping.
@struct.tool()
async def search(query: str):
    ...

# Required: wrap the agent loop yourself.
async with struct.agent(name="checkout"):
    msg = await client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=1024,
        messages=[...],
    )
    # Dispatching a decorated tool inside the agent emits a fully-populated
    # execute_tool span (name, id, arguments, result):
    result = await search(query="...")

For dynamic dispatch (the LLM picks a tool from a registry at runtime), apply the decorator at runtime — still automatic, just bind the name when you wrap the callable:

registry = {t.name: struct.tool(name=t.name)(t.execute) for t in tools}
result = await registry[block.name](**block.input)   # arguments + result captured

struct.tool() can also be used as a context manager (async with struct.tool(name=...): ...) to instrument an arbitrary block of code as a tool span. That form is a manual escape hatch — it does NOT auto-capture arguments/result (a with block can't see the body's return value), so prefer the decorator for actual tool calls. See Parallel tool calls for the one runtime value (tool_call_id) you must supply under concurrency.

anthropic.Anthropic, anthropic.AsyncAnthropic, and the bedrock/vertex clients are all auto-instrumented for chat spans.

MCP

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="support-agent")

from mcp import ClientSession
async with struct.agent(name="support", session_id=conversation_id):
    result = await session.call_tool("get_weather", {"city": "Paris"})

Every ClientSession.call_tool (mcp 1.x and 2.x, including the OpenAI Agents SDK's MCP servers) becomes a tools/call get_weather CLIENT span with mcp.method.name, gen_ai.operation.name = execute_tool and the request's jsonrpc.request.id. Its trace context is added to the request's params._meta (traceparent, tracestate, baggage), so an instrumented MCP server continues the same trace. On mcp 1.x, trace keys you pass in meta win; mcp 2.x writes the trace keys again itself on every transport but its in-process Client, from its own MCP send span when your global tracer provider records (that span is a child of Struct's, so the server stays in the trace). mcp before 1.19 has no meta parameter: those calls get the span without propagation.

The first MCP call inside a struct.tool() scope is recorded on that tool span, including its outcome; later calls get child spans. An isError result is error.type = tool_error; a JSON-RPC error records its code in error.type and rpc.response.status_code.

Parallel tool calls — pass tool_call_id explicitly

When you execute an assistant turn's tool calls sequentially — one await at a time, in the order the tool_use blocks appear — struct.tool() auto-fills gen_ai.tool.call.id by matching each span to the next pending tool_use of the same tool name. Nothing extra to do.

When you execute them concurrently (e.g. asyncio.gather), that name-and-order matching is ambiguous: two struct.tool(name="search") spans can start in any order, so the auto-fill may attach the wrong id (and thus the wrong arguments/result) to a call. In that case pass tool_call_id explicitly from the originating tool_use block — an explicit id always overrides the auto-linkage:

async def run_one(block):
    # The id from THIS block overrides the name/order auto-fill.
    async with struct.tool(name=block.name, tool_call_id=block.id):
        return await dispatch(block.name, **block.input)

# Concurrent execution — each tool span still carries the correct id.
results = await asyncio.gather(*[run_one(b) for b in tool_use_blocks])

Rule of thumb: serial tool execution → automatic; concurrent tool execution → provide tool_call_id= yourself. (Auto-instrumented frameworks such as LangChain read the id from the framework's ToolCall, so this only applies when you drive the tool loop directly against an LLM SDK.)

LangChain BaseChatModel (no agent/graph)

If you call ChatAnthropic.invoke(...) (or any other BaseChatModel) without wrapping it in an AgentExecutor or LangGraph, only the chat span emits automatically. Same rule as raw Anthropic — wrap your agent loop in struct.agent() and tool execution in struct.tool().

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="my-agent")

from langchain_anthropic import ChatAnthropic
llm = ChatAnthropic(model="claude-3-5-sonnet-20241022")

async with struct.agent(name="my-agent"):
    response = await llm.ainvoke([("user", "...")])
    async with struct.tool(name="search"):
        ...

When you do use ChatAnthropic and have the anthropic SDK installed, the chat span comes from the LangChain layer (single span); the Anthropic instrumentation skips its own to avoid duplicates.

LiteLLM

from struct_sdk import struct
struct.init(ingest_key="pk-...", service_name="my-agent")

import litellm  # wrapped on import; a `completion` imported before init() is not traced

async with struct.agent(name="my-agent"):
    response = await litellm.acompletion(model="anthropic/claude-sonnet-5", messages=[...])
    async with struct.tool(name="search"):
        ...

Each call is one chat {model} span, whichever provider LiteLLM routes to. A framework that owns the call (LangChain, pydantic-ai, the OpenAI Agents SDK) keeps its own span and LiteLLM adds none. LiteLLM's own litellm.callbacks = ["otel"] would add a second, LiteLLM-shaped span; leave it off when Struct is on. struct.init() does not import LiteLLM (its import fetches the model cost map over the network): it wraps litellm.completion and litellm.acompletion when your code imports litellm. pip install "struct-sdk[litellm]" installs LiteLLM with its mcp extra, which recent LiteLLM releases need for calls that pass tools.

Content capture

The SDK supports four capture modes controlling how prompt/response content is emitted.

from struct_sdk import struct, ContentCaptureMode

struct.init(
    ingest_key=...,
    content_capture=ContentCaptureMode.EVENT_ONLY,  # opt in; the default is NONE
    # or ContentCaptureMode.SPAN_ONLY, SPAN_AND_EVENT
)

• EVENT_ONLY: each model call's content lands on one OTel log record, gen_ai.client.inference.operation.details (input and output messages, system instructions, tool definitions, as structured values). Spans carry metadata only.
• SPAN_ONLY: content on span attributes (gen_ai.input.messages, gen_ai.output.messages).
• SPAN_AND_EVENT: both.
• NONE (default since 0.3.0): no content captured, including the user prompt on the agent span and tool arguments and results. Token counts, tool call IDs, finish reasons, and other metadata still flow.

capture_content=False is a shorthand for ContentCaptureMode.NONE, and capture_content=True for EVENT_ONLY.

Manual scopes

struct.agent() and struct.tool() create invoke_agent and execute_tool spans. Use them when you call an LLM SDK directly; you don't need them when an agent framework (LangGraph, AgentExecutor, Claude Agent SDK) is already creating those spans for you.

async with struct.agent(
    name="onboarding",
    session_id=conversation_id,
    metadata={"tenant": "acme"},
):
    async with struct.tool(name="fetch-profile"):
        return await fetch_profile()

Decorator form:

@struct.agent(name="checkout")
async def run_checkout(order_id: str):
    @struct.tool(name="charge-card")
    async def charge():
        return await stripe.charge(order_id)

    return await charge()

Sub-agents (e.g. a create_agent graph invoked from inside another agent's tool body) record their parent via the struct.agent.parent_session_id attribute on the inner invoke_agent span. This powers the UI's "Spawned by" backlink, which works for any nested invocation.

The parent's "Subagents" forward list — the inverse direction — additionally requires that the nested invoke shares the outer agent's trace. This works automatically when the outer tool is built with the @tool decorator (or any callback-aware wrapping) since the nested invoke inherits the parent's run state. Bare Tool(...) constructors that bypass the callback chain can break the forward link; the backlink still renders.

Semantic conventions

Emits attributes per the OTel GenAI semantic conventions:

• gen_ai.operation.name — chat, execute_tool, invoke_agent, retrieval
• gen_ai.provider.name — the GenAI provider on inference spans (anthropic, openai, …); platform-routed calls report the platform value (aws.bedrock, gcp.vertex_ai, azure.ai.openai) when detectable from the client. invoke_agent spans inherit the provider from their first child inference call (omitted when no model is reached); execute_tool/retrieval spans carry no provider.
• gen_ai.request.{model, max_tokens, temperature, top_p, top_k, stop_sequences}
• gen_ai.response.{model, id, finish_reasons}
• gen_ai.usage.{input_tokens, output_tokens, cache_read.input_tokens, cache_creation.input_tokens}
• gen_ai.conversation.id
• gen_ai.tool.{name, call.id, call.arguments, call.result}
• error.type + StatusCode.ERROR on failures

error.type values emitted

The OTel conventions ask instrumentations to document the error values they report ("Instrumentations SHOULD document the list of errors they report"). This SDK emits exactly four kinds of value, all alongside span StatusCode.ERROR:

tool_error is a deliberate low-cardinality sentinel, which the error.type convention explicitly permits ("another low-cardinality error identifier"; a custom value MAY be used where no well-known one applies). The same split is used by OpenTelemetry's MCP instrumentation in OpenLLMetry, which likewise reports error.type="tool_error" for the isError path and the exception class name otherwise.

The distinction is what lets a monitor page on genuine execution failures while excluding failures the model already saw and can recover from.

Note: gen_ai.usage.input_tokens for Anthropic is the true total — the SDK adds back cache_read_input_tokens and cache_creation_input_tokens, which Anthropic's raw response excludes from input_tokens.

Configuration

struct.init(
    ingest_key="pk-...",                # required
    service_name="my-agent",            # default: OTEL_SERVICE_NAME, else "default-agent"
    service_version="1.2.3",            # default: from OTEL_RESOURCE_ATTRIBUTES, else "0.0.0"
    environment="production",           # default: from OTEL_RESOURCE_ATTRIBUTES, else "development"
    endpoint="https://ingest.struct.ai", # default; override for self-hosted
    shutdown_timeout_seconds=5.0,        # default: 5.0
    content_capture=ContentCaptureMode.EVENT_ONLY,  # default: NONE
    auto_instrument=True,                # default; False = AI libraries only
    instrumentation_options=None,        # per-library full-app settings
    redact_attributes=["*.password"],    # default: STRUCT_REDACT_ATTRIBUTES
)

With auto_instrument=False the SDK uses an isolated TracerProvider and LoggerProvider and instruments only the AI libraries; your existing OTel setup is unaffected. The default is full-app mode.

Service identity follows the standard OpenTelemetry variables when you don't pass it: OTEL_SERVICE_NAME, and service.version and deployment.environment in OTEL_RESOURCE_ATTRIBUTES. Arguments to init() win. The resource also carries host, runtime and (with the [auto] extra) container metadata, plus the commit when STRUCT_GIT_COMMIT_SHA and STRUCT_GIT_REPOSITORY_URL are set. It includes a service.instance.id so replicas stay distinguishable; an ID from OTEL_RESOURCE_ATTRIBUTES is preserved.

In pre-fork servers (for example gunicorn --preload), call struct.init() in each worker (gunicorn's post_fork hook), not in the parent: forked workers would otherwise share the parent's instance id. A worker that exports after being forked from an initialized parent logs one INFO line saying so.

Redaction

redact_attributes takes glob patterns, matched case-insensitively. Span, span-event and log attributes whose keys match leave the process as [REDACTED], for example ["http.request.header.authorization", "*.password"]. STRUCT_REDACT_ATTRIBUTES (comma-separated) sets them without code. Only what Struct sends is changed; your own exporters still see the real values. Metrics, log bodies and span names are not redacted.

Full-app mode

Full-app mode is the default since 0.3.0: struct.init() adds APM-style telemetry for the whole service, like an OpenTelemetry distro: HTTP servers and clients, databases, caches, queues, app logs and process metrics, built on the OpenTelemetry contrib instrumentors. Every supported library you have installed is instrumented; turn individual ones off below. To keep the 0.2 behavior (AI libraries only, isolated providers), pass auto_instrument=False.

pip install "struct-sdk[auto]"           # needs opentelemetry-sdk 1.30+; add [azure] for Azure SDK tracing,
                                         # [structlog] for structlog (opentelemetry-sdk 1.44+),
                                         # [exceptions] for uncaught exceptions (1.43+)

import fastapi
from struct_sdk import struct

struct.init(ingest_key="pk-...", service_name="checkout-api")

app = fastapi.FastAPI()  # resolve the class after init()

Framework instrumentors replace the framework's class when init() runs, so an app class bound earlier stays uninstrumented: from fastapi import FastAPI above struct.init() keeps the original class. Import framework classes after init(), or reference them through the module as above. For an app created before init(), call the contrib instrumentor directly (for example FastAPIInstrumentor.instrument_app(app)). In pre-fork servers, run init() in each worker, as described above.

Each instrumentor activates only when its library is installed. instrumentation_options disables or configures one by name; other keys are passed to that instrumentor:

struct.init(
    ingest_key="pk-...",
    instrumentation_options={
        "fastapi": {"excluded_urls": "health,ready"},
        "logging": {"enabled": False},
    },
)

As with opentelemetry-instrument and the Azure Monitor distro, the standard OTEL_PYTHON_DISABLED_INSTRUMENTATIONS environment variable also turns libraries off (OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=requests,urllib3, or * for all of them). An explicit instrumentation_options entry takes precedence over it.

Full-app mode installs Struct's providers as the global OpenTelemetry providers only when no tracer, meter or logger provider is installed. If the process already configured any of them (Azure Monitor, your own SDK setup), Struct adds its exporters to your SDK tracer and logger providers and instruments nothing: your setup keeps ownership of instrumentation, sampling and shutdown, and your metrics are not sent to Struct. Struct's own GenAI spans keep their isolated provider either way, so your sampler does not drop them.

Datadog ddtrace

Under ddtrace-run, Struct leaves the libraries ddtrace patches to it, and urllib3 beneath requests or botocore, to avoid duplicate spans and conflicting trace headers, and logs one INFO line listing them. Logs, process metrics, thread context, AI spans and libraries ddtrace doesn't patch still go to Struct. With DD_TRACE_OTEL_ENABLED=true, ddtrace's OpenTelemetry provider isn't the OpenTelemetry SDK, so full-app mode attaches nothing (an INFO line says so) and only AI spans reach Struct.

Zero-code activation

OTEL_PYTHON_DISTRO=struct \
STRUCT_INGEST_KEY=pk-... \
OTEL_SERVICE_NAME=checkout-api \
opentelemetry-instrument python app.py

opentelemetry-instrument comes with [auto]. Struct calls struct.init() before your app starts, with STRUCT_INGEST_URL as the endpoint when set. Message content stays off; call struct.init() in code to turn it on. Installing the SDK never changes another distro: Struct's distro is used only when OTEL_PYTHON_DISTRO=struct. When selected, it also keeps other installed configurators (such as opentelemetry-distro's) from installing competing providers, unless you set OTEL_PYTHON_CONFIGURATOR yourself. If you already run another distro, call struct.init() in code instead and Struct attaches to its providers.

Temporal

pip install "struct-sdk[temporal]"

from temporalio.client import Client
from temporalio.runtime import Runtime
from struct_sdk import struct, temporal

struct.init(ingest_key="pk-...", service_name="billing-worker")
runtime = Runtime(telemetry=temporal.telemetry_config(global_tags={"replica": replica_id}))
client = await Client.connect(
    "temporal:7233",
    runtime=runtime,
    interceptors=[temporal.tracing_interceptor()],
)

telemetry_config() exports Temporal Core metrics (task slots, schedule-to-start latency, poll results, sticky cache) to Struct. It keeps Core's console logging at Temporal's defaults, except that failed metric exports are not printed. tracing_interceptor() (temporalio 1.11+) traces workflows and activities with Struct's tracer; a failing span processor never blocks a Temporal call or replaces its exception. Temporal server metrics are a Prometheus endpoint; scrape them with an OpenTelemetry Collector and export OTLP/HTTP to your Struct ingest endpoint with the x-struct-ingest-key header.

Google Cloud clients

from struct_sdk import gcp, struct

struct.init(ingest_key="pk-...", service_name="monitor-worker")
gcp.instrument()  # once at startup, before creating Google clients

Released gRPC Google clients (Cloud Logging, Cloud Monitoring, Secret Manager, Pub/Sub admin and the other generated libraries) emit no OpenTelemetry spans. gcp.instrument() (no extra needed) gives each unary RPC attempt one CLIENT span, google.monitoring.v3.MetricService/ListTimeSeries, with rpc.service, rpc.method, rpc.grpc.status_code and server.address; a retried call shows one span per attempt. Streaming RPCs are passed through untouched. Your metadata keeps repeated keys; the trace headers of your configured propagator replace any copies of its fields, and trace fields gRPC would reject are dropped. A failure in Struct's tracing never affects the call. Spans go to the global tracer provider (Struct's, or yours when Struct attaches to your OpenTelemetry setup, so your exporter gets them too), or to tracer_provider= if you pass one. Channels already traced by contrib's gRPC instrumentor keep its provider and behavior. struct.init() never calls it for you. Clients created before the call, and transports you give your own channel, are left alone. OTEL_PYTHON_DISABLED_INSTRUMENTATIONS=gcp turns it off and OTEL_PYTHON_GRPC_EXCLUDED_SERVICES excludes services; under a ddtrace that patches grpc it steps aside.

Other Google libraries trace themselves: google-cloud-bigquery emits its own BigQuery.* spans whenever opentelemetry-instrumentation is installed (its path and db.name attributes carry project, dataset and table names, never query text). google-cloud-storage needs ENABLE_GCS_PYTHON_CLIENT_OTEL_TRACES=True set before it is imported, and the Pub/Sub publisher and subscriber need enable_open_telemetry_tracing=True in their options. google-api-python-client (discovery) calls get only the HTTP spans of an instrumented transport.

Reliability

The SDK is designed never to break your application. Instrumentation hooks, span exports, and shutdown all swallow exceptions internally and log at DEBUG on the struct_sdk logger, with a traceback for the first failure at each site. Process exit is bounded by shutdown_timeout_seconds and runs in a background thread, so a slow or unreachable ingest endpoint cannot hang shutdown.

Troubleshooting

• Spans missing after instrumenting: Call struct.init() before importing the instrumented libraries, so the SDK can wire up instrumentation before any instance is constructed.
• No message log records appearing: they only emit when the capture mode is EVENT_ONLY or SPAN_AND_EVENT. The default is NONE.
• Duplicate chat spans: When an LLM provider SDK and a wrapping framework are both instrumented (e.g. ChatAnthropic calling through to anthropic), the provider-level chat span is skipped and the framework's is kept.

License

Apache-2.0