otelzero
Instrument agent frameworks via Python import hooks and stream telemetry to a
Kiji Agent Registry server — no code changes to your
agents required. (Import package: otelzero.)
Site: 575-lab.github.io/otelzero — an animated walkthrough of the import hook, with live Kiji and Dataiku Agent Management demos.
Install monitoring for your framework before importing it, then run as usual:
from otelzero import install_crewai_monitoring
install_crewai_monitoring(server_url="http://localhost:8000")
from crewai import Agent, Crew, Task # now auto-instrumented
Order matters: install the hook before importing the framework. The hooks use the
find_specimport protocol and require Python ≥ 3.12.
Supported SDKs / hooks
For exactly which agent actions each framework reports (agent runs, model calls, tool calls, workflows) and whether they come from the framework's own GenAI telemetry or from an otelzero hook, see FRAMEWORKS.md. For the span attributes each framework emits, see the attribute matrix.
| Framework | Install function | What gets wrapped |
|---|---|---|
| CrewAI | install_crewai_monitoring |
Crew.kickoff, Agent.execute_task, Task.execute_sync |
| LangChain | install_langchain_monitoring |
nothing — switches on LangSmith's native OpenTelemetry export (translated into the current GenAI conventions on export) |
| LangGraph | install_langgraph_monitoring |
nothing — same LangSmith export; one run per graph run, subgraphs included in it |
| LlamaIndex | install_llama_index_monitoring |
Workflow.run (covers FunctionAgent, ReActAgent, CodeActAgent, AgentWorkflow and hand-written workflows) |
| OpenAI Agents SDK | install_openai_agents_monitoring |
Runner.run, Runner.run_sync, Runner.run_streamed |
| Pydantic AI | install_pydantic_ai_monitoring |
nothing — configures Pydantic AI's native OpenTelemetry (Agent.instrument_all) |
| smolagents | install_smolagents_monitoring |
MultiStepAgent.run (covers CodeAgent and ToolCallingAgent, direct + streaming) |
| AWS Strands Agents | install_strands_monitoring |
nothing — points Strands' native OpenTelemetry at the registry (latest GenAI conventions) |
| Google ADK | install_adk_monitoring |
nothing — points ADK's native OpenTelemetry at the registry (latest GenAI conventions) |
| Databricks Agent Bricks / Mosaic AI | install_databricks_monitoring |
each ResponsesAgent / ChatAgent subclass's predict, plus its tool calls |
| Claude Agent SDK | install_claude_agent_monitoring |
query, ClaudeSDKClient.receive_response (+ ClaudeSDKClient.query, for its prompt) |
| Microsoft Agent Framework | install_microsoft_monitoring |
nothing — switches on Agent Framework's native OpenTelemetry (enable_instrumentation) |
| Snowflake Cortex Agent SDK | install_snowflake_monitoring |
query, CortexCodeSDKClient.receive_response (+ CortexCodeSDKClient.query, for its prompt) |
| Anything else | install_monitoring |
framework-agnostic manual instrumentation (TelemetryClient) |
Native-OTel frameworks. Pydantic AI, AWS Strands Agents, Google ADK and
Microsoft Agent Framework already trace themselves in the
GenAI semantic conventions,
so their install functions wrap nothing. Each one switches on the framework's
own instrumentation, exports it to the registry's OTLP receiver, stamps
otelzero's run identity on every span, and derives the usual agent_start /
agent_end / agent_error events from the framework's invoke_agent spans.
LangChain and LangGraph work the same way through LangSmith's OpenTelemetry
export, whose spans otelzero translates into the current conventions on
export, with the events coming
from LangChain's own callbacks (see
the LangGraph guide).
For all of these, the import order doesn't matter. The otel extra is required,
and their tracing stays on even with OTELZERO_OTEL=0.
LangChain 1.0 note: create_agent (and the older create_react_agent) return
a compiled LangGraph graph. Both installers set up the same LangSmith export, so
either one captures it. Installing both is safe and still reports one run per
graph run.
Every hook streams the same event envelope (agent_start / agent_end /
agent_error, plus crew/task events where the framework has them) to the otelzero
dashboard. With the otel extra installed, the hooks also emit standard
OpenTelemetry spans — tracing is on by default (OTELZERO_OTEL is
effectively always 1; set OTELZERO_OTEL=0 to switch the import hooks' spans
off) — and a framework's own native OTel spans (model/tool calls) nest
underneath otelzero's. Set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1
to also record each run's prompt and response as gen_ai.input.messages /
gen_ai.output.messages (on the span and in the dashboard's event details);
off by default, capped by OTELZERO_CONTENT_MAX_CHARS (default 4000 per part).
Install
pip install otelzero # core client
pip install "otelzero[otel]" # + OpenTelemetry span export
Set up a server
The client needs a Kiji Agent Registry server (dashboard + telemetry ingest) to send events to.
Self-hosted (minimal): install the kiji-agent-registry distribution and run it:
pip install kiji-agent-registry # add [otel] for the OTLP ingest endpoint
kiji-agent-registry # dashboard + ingest on http://localhost:8000
In your own cloud: marketplace/ in the project repo packages the server for the
GCP, AWS and Azure marketplaces — one VM plus a data disk inside your own project,
account or subscription. The listings are still in publisher review; until they land,
the Docker Compose and container paths in the project README cover the same ground.
Point the client at whatever URL your deployment serves:
install_crewai_monitoring(
server_url="https://kiji.your-vpc.internal",
api_key="your-ingest-key", # or set OTELZERO_API_KEY
)
Managed: Dataiku Agent Management runs it for you — agent registry, drift detection and governance.
Ingest is owner-scoped: pass an ingest key via
install_*_monitoring(..., api_key="...") or the OTELZERO_API_KEY env var (any
string ≥ 8 chars auto-provisions its own owner on a fresh server).
Docker images, OTLP/collector ingest, and the mobile companion apps are covered in the project README.
Try it against a local registry
Run a Kiji Agent Registry next to this checkout, then drive it with the framework-free simulator:
git clone https://github.com/575-lab/kiji-agent-registry ../kiji-agent-registry
(cd ../kiji-agent-registry && uv run kiji-agent-registry) # terminal 1
uv run python demo/demo_simulator.py # terminal 2
Open http://localhost:8000 to watch the runs arrive. The commands below assume
the registry checkout sits next to this one at ../kiji-agent-registry.
Framework guides
Instrument a real CrewAI crew
The import hook wraps the real crewai package, so install monitoring
before importing CrewAI — then write your crew normally, with no telemetry
code in it:
from otelzero import install_crewai_monitoring
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
install_crewai_monitoring(server_url="http://localhost:8000", user="Sarah")
# 2) Now import CrewAI — Crew/Agent/Task are auto-instrumented as they load
from crewai import Agent, Crew, Task
researcher = Agent(role="Research Analyst", goal="...", backstory="...")
task = Task(description="...", expected_output="...", agent=researcher)
crew = Crew(agents=[researcher], tasks=[task])
crew.kickoff() # streams crew/agent/task events to the dashboard automatically
A complete, runnable version is in demo/demo_crewai.py:
uv pip install crewai # the hook instruments the real package
export OPENAI_API_KEY=... # for a full kickoff
uv run python demo/demo_crewai.py
Order matters: instrumentation must be installed before
crewaiis imported. The hooks use thefind_specprotocol and require Python ≥ 3.12.
Instrument a real LangGraph graph (and LangChain 1.0 agents)
LangChain and LangGraph trace themselves through LangSmith, and the
langsmith SDK can export those traces as OpenTelemetry spans
(langsmith[otel]). Every graph run, node, subgraph, chain, model and tool call
becomes one span, named after the run (Research Graph, model, tools, a
tool's name). It carries langsmith.span.kind, gen_ai.operation.name,
gen_ai.request.model, gen_ai.usage.* and langsmith.metadata.*.
install_langgraph_monitoring and install_langchain_monitoring wrap nothing.
They:
- make otelzero's OTLP provider the global one, which is where LangSmith's
exporter sends its spans. On their way out they pass through otelzero's
translator (below):
the root run becomes
invoke_agent {run}, model callschat {model}, tool callsexecute_tool {tool}. If your application set its own global tracer provider first, they go out through that one, untranslated; - turn the export on through LangSmith's own variables:
LANGSMITH_TRACING,LANGSMITH_OTEL_ENABLEDand, unless a LangSmith API key is configured,LANGSMITH_OTEL_ONLY(so nothing goes to LangSmith's cloud). Unless content capture is on, they also setLANGSMITH_HIDE_INPUTS/LANGSMITH_HIDE_OUTPUTS(LangSmith otherwise exports every prompt and completion). Values you set yourself are kept, and an uninstall removes only what was added; - register a callback handler through LangChain's own
register_configure_hook. LangChain adds it to every run the way it adds its own tracers. It sends oneagent_start/agent_end/agent_errorper top-level run (the outermost graph, agent, chain or runnable you invoked), with the model, the tools it called and its token usage collected from the model and tool runs under it. Nodes and subgraphs are part of the run, so one graph run is one agent card. The session is titled from the run's input.
from otelzero import install_langgraph_monitoring
from langgraph.graph import END, START, MessagesState, StateGraph # any order
install_langgraph_monitoring(server_url="http://localhost:8000", user="Sarah")
graph = StateGraph(MessagesState)
graph.add_node("agent", call_model)
graph.add_edge(START, "agent")
graph.add_edge("agent", END)
app = graph.compile(name="Research Graph")
app.invoke({"messages": [("user", "who won?")]}) # one agent card per run
Why the events come from callbacks. LangSmith builds its spans on a background thread, when it flushes its queue. A finished run's span is created and ended in one go, before its children's spans exist. The callbacks fire in your code's context, as the run happens, so the events are live.
Identity. A run names its own thread and user in its metadata:
config={"metadata": {"thread_id": ..., "user_name": ...}} (also
conversation_id / session_id, user_id / user), or LangGraph's
config={"configurable": {"thread_id": ...}}. Its events and its spans both
use that, so the dashboard groups them together. Without metadata, the events
fall back to the active run_context(), then to the client's user= /
session=. The spans fall back to the client's only: a span can't be tied
back to the run it records, so run_context() doesn't reach them.
Short scripts: call langchain_core.tracers.langchain.wait_for_all_tracers()
before otel.force_flush() at exit, so LangSmith's queue is drained into spans
first.
Token usage comes from the model calls a run makes (their LLMResult). A graph
node that returns prepared messages without calling a model reports none.
A complete, runnable version is in demo/demo_langgraph.py (fully offline —
its nodes return prepared replies instead of calling a model, and it contains a
subgraph so you can see it counted as part of its parent's run):
uv pip install langgraph "langsmith[otel]" "otelzero[otel]"
uv run python demo/demo_langgraph.py
Instrument a real OpenAI Agents SDK agent
The OpenAI Agents SDK (the
openai-agents package, OpenAI's production successor to the archived Swarm)
starts every run through one of three Runner classmethods, and
install_openai_agents_monitoring wraps all three — run, run_sync and
run_streamed — so a run is captured exactly once whichever you use:
from otelzero import install_openai_agents_monitoring
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
install_openai_agents_monitoring(server_url="http://localhost:8000", user="Sarah")
# 2) Now import the SDK — Runner is auto-instrumented as it loads
from agents import Agent, Runner
agent = Agent(name="Weather Agent", instructions="...", model="gpt-4o-mini")
await Runner.run(agent, "What is the weather in Amsterdam?")
run_sync drives run underneath, so both are wrapped but only the outer call
opens a span. A streamed run is traced across its stream_events() pass, since
that is when the run is actually delivered.
The agent_start details carry the run's model, tools and handoffs; token
counts come from the run context's usage. A complete, runnable version is in
demo/demo_openai_agents.py:
uv pip install openai-agents # the hook instruments the real package
export OPENAI_API_KEY=... # the SDK has no offline model
uv run python demo/demo_openai_agents.py
Order matters: instrumentation must be installed before
agentsis imported. The hooks use thefind_specprotocol and require Python ≥ 3.12.
Instrument a real Pydantic AI agent
Pydantic AI ships its own OpenTelemetry
instrumentation, already in the GenAI semantic conventions: one
invoke_agent {name} span per run with chat and execute_tool children,
gen_ai.* usage attributes and opt-in message content. That is richer than a
wrapper around Agent.run could see from the outside, so
install_pydantic_ai_monitoring wraps nothing — it points the native
instrumentation (Agent.instrument_all) at the registry's OTLP receiver,
stamps otelzero's run identity (gen_ai.conversation.id, user.name,
otelzero.session.name) onto every span, and turns each native invoke_agent
span into the usual agent_start / agent_end / agent_error events:
from otelzero import install_pydantic_ai_monitoring
from pydantic_ai import Agent # import order does not matter for this one
install_pydantic_ai_monitoring(
server_url="http://localhost:8000",
user="Sarah",
name="Order support triage", # session title, on every event and span
)
agent = Agent("openai:gpt-4o", name="Support Agent")
result = await agent.run("What is the status of order A-1042?")
async with agent.run_stream("...") as stream: # streaming runs too
...
The events are derived from the spans, so this hook needs the otel extra, and
its tracing stays on even with OTELZERO_OTEL=0. They carry what
Pydantic AI has recorded: model at start; tools and usage at the end;
and on failure the exception message and traceback only when content capture is
on (otherwise just the exception type). Pydantic AI's spans hold no prompt when
a run starts, so — unlike the wrapping hooks — no session title is derived from
it: pass name= (or run_context(name=...)) to title the session. Prompts and
completions follow the usual OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT
opt-in; pass settings=InstrumentationSettings(...) to choose a different
format version or content policy.
A complete, runnable version is in demo/demo_pydantic_ai.py (fully offline —
it uses Pydantic AI's own TestModel, so no API key is needed):
uv pip install pydantic-ai "otelzero[otel]"
uv run python demo/demo_pydantic_ai.py
uv run python demo/demo_pydantic_ai.py --model openai:gpt-4o # a real model
Instrument a real smolagents agent
Hugging Face's smolagents builds
both of its agent flavours — CodeAgent, which writes its actions as Python, and
ToolCallingAgent, which emits structured tool calls — on one MultiStepAgent
base, and neither overrides run. So install_smolagents_monitoring wraps that
single method and covers every agent the library can build:
from otelzero import install_smolagents_monitoring
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
install_smolagents_monitoring(server_url="http://localhost:8000", user="Sarah")
# 2) Now import smolagents — MultiStepAgent.run is auto-instrumented as it loads
from smolagents import CodeAgent, InferenceClientModel
agent = CodeAgent(tools=[...], model=InferenceClientModel(), name="Travel Agent")
agent.run("How long is the drive from Paris to Lyon?") # streams agent events
for step in agent.run("...", stream=True): # the streaming shape too
...
run is a synchronous dispatcher whose stream argument decides its return
shape — the final answer, or a generator of steps — and otelzero traces whichever
it is. Unlike a framework that hands back an awaitable, the direct call performs
the whole run before returning, so the streaming decision is read off the
argument rather than the returned object.
Token counts come from the agent's own monitor, which totals them across the
runs it has seen. otelzero reports the difference across each run, so a
continuation (agent.run(..., reset=False)) is charged only for what it actually
spent rather than for the whole conversation so far.
A complete, runnable version is in demo/demo_smolagents.py:
uv pip install smolagents # the hook instruments the real package
export HF_TOKEN=... # the default InferenceClientModel needs it
uv run python demo/demo_smolagents.py
Order matters: instrumentation must be installed before
smolagentsis imported. The hooks use thefind_specprotocol and require Python ≥ 3.12.
Instrument a real LlamaIndex agent
Every modern LlamaIndex agent —
FunctionAgent, ReActAgent, CodeActAgent, the multi-agent AgentWorkflow —
is a workflow: each overrides run only to package its arguments into a start
event and then calls Workflow.run. So install_llama_index_monitoring wraps
that one method and covers every agent the library can build, plus any workflow
you write by hand:
from otelzero import install_llama_index_monitoring
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
install_llama_index_monitoring(server_url="http://localhost:8000", user="Sarah")
# 2) Now import LlamaIndex — Workflow.run is auto-instrumented as it loads
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI
agent = FunctionAgent(name="Support Agent", tools=[lookup_order], llm=OpenAI(model="gpt-4o-mini"))
await agent.run(user_msg="What is the status of order A-1042?") # streams agent events
Workflow.run is synchronous: it schedules an asyncio task and hands back a
WorkflowHandler while the run continues in the background. otelzero therefore
opens the run when run() is called and closes it from a done-callback on that
task — so a run is captured whether you await the handler, read only
stream_events(), or cancel it with cancel_run() (which closes the run as
interrupted rather than leaving it dangling). An agent or workflow started from
inside a tool folds into the run it belongs to, the way a LangGraph subgraph is
part of its parent's run.
Token counts do not ride on the result — AgentOutput has no usage — so otelzero
registers one handler on LlamaIndex's own instrumentation dispatcher and sums the
LLMChatEndEvent / LLMCompletionEndEvent payloads a run produces. Predict and
structured-predict events are deliberately not counted: each wraps an inner chat
call that is already counted, and a CustomLLM that reports one payload through
two events is counted once.
Three limitations worth knowing:
- a human-in-the-loop run that is never resumed never closes, so its card stays open;
- OpenAI streaming reports no tokens unless the model is built with
additional_kwargs={"stream_options": {"include_usage": True}}(orstreaming=False) — andFunctionAgentstreams by default; - LLM events that arrive after a run has closed are dropped.
A complete, runnable version is in demo/demo_llama_index.py:
uv pip install llama-index-core # the hook instruments the real package
uv run python demo/demo_llama_index.py # fully offline
uv run python demo/demo_llama_index.py --model gpt-4o-mini # a real model
The offline run uses LlamaIndex's own MockFunctionCallingLLM, which returns no
provider payload — and a provider payload is where token counts live — so the
offline cards show runs without tokens. --model gives real counts.
Order matters: instrumentation must be installed before
llama_indexis imported. The hooks use thefind_specprotocol and require Python ≥ 3.12.
Instrument a real AWS Strands Agents agent
AWS Strands Agents always traces
itself through the global OpenTelemetry tracer provider, in the GenAI semantic
conventions. Each run gets an invoke_agent {name} span, with
execute_event_loop_cycle, chat and execute_tool {tool} spans under it,
plus gen_ai.* model, tool and usage attributes. install_strands_monitoring
wraps nothing. It:
- makes otelzero's OTLP provider the global one, so Strands' spans export to the
registry's
/v1/traces; - opts Strands into the latest GenAI conventions
(
OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental:gen_ai.provider.name,gen_ai.input.messages/gen_ai.output.messages); - applies otelzero's content opt-in. By default Strands records prompts and
completions unconditionally. With content capture off, the hook turns on
Strands' redaction (
gen_ai_unredacted_attributes=). WithOTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1, the content goes on span attributes (gen_ai_span_attributes_only) together with tool definitions; - stamps otelzero's run identity on every Strands span, and turns each
invoke_agentspan intoagent_start/agent_end/agent_error.
Opt-in tokens you set yourself are kept, and an uninstall removes only the ones the hook added.
from otelzero import install_strands_monitoring
from strands import Agent, tool # import order does not matter for this one
install_strands_monitoring(
server_url="http://localhost:8000",
user="Sarah",
name="Calculator session", # session title, on every event and span
)
@tool
def add(a: int, b: int) -> int:
"""Add two integers."""
return a + b
agent = Agent(name="Calculator Agent", tools=[add])
agent("What is 21 + 21?") # Strands' own spans -> registry; agent_* events derived from them
Strands' agent span carries no prompt when a run starts, so no session title
is derived from the prompt. Pass name= (or run_context(name=...)) to set
one.
A complete, runnable version is in demo/demo_strands.py:
uv pip install strands-agents "otelzero[otel]"
# configure AWS Bedrock credentials for a full run (Strands' default model)
uv run python demo/demo_strands.py
Instrument a real Google ADK agent
Google ADK (Agent Development Kit) also
traces itself through the global tracer provider. A Runner run opens an
invoke_workflow {app} span. Each agent that takes a turn gets an
invoke_agent {name} span under it, with generate_content {model} and
execute_tool {tool} spans nested inside. install_adk_monitoring wraps
nothing. It makes otelzero's OTLP provider the global one, opts ADK into the
latest GenAI conventions (gen_ai_latest_experimental, which replaces ADK's
legacy gcp.vertex.agent.* payload attributes), stamps user.name and
otelzero.session.name on every ADK span, and turns each invoke_agent span
into agent_start / agent_end / agent_error. A multi-agent run reports
each agent that takes a turn.
from otelzero import install_adk_monitoring
from google.adk.agents import Agent # import order does not matter
from google.adk.runners import InMemoryRunner
install_adk_monitoring(server_url="http://localhost:8000", user="Sarah")
agent = Agent(name="calculator_agent", model="gemini-3.7-flash", tools=[...])
runner = InMemoryRunner(agent=agent, app_name="my-app")
# Each runner.run / run_async: ADK's own spans -> registry, agent_* events from them
Session. ADK already writes its own session id as gen_ai.conversation.id,
after each span starts, so otelzero can't replace it. It is also the right
value: an ADK session is the conversation, and you choose it when you call
run_async(session_id=...). So ADK runs are grouped by ADK's session id, the
events are filed under it too, and the session= argument only applies to a
run whose spans carry none.
Content. ADK reads OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT on
its own terms. 1 / true mean EVENT_ONLY there (log records, not spans).
Set SPAN_ONLY or SPAN_AND_EVENT to put prompts and completions on the spans
the registry ingests.
A complete, runnable version is in demo/demo_adk.py:
uv pip install google-adk "otelzero[otel]"
export GOOGLE_API_KEY=... # for a full run (ADK's default Gemini model)
uv run python demo/demo_adk.py
Instrument a real Databricks Agent Bricks / Mosaic AI agent
Databricks agents (Agent Bricks,
the Mosaic AI Agent Framework) are authored as MLflow ResponsesAgent
subclasses (the current recommended interface; the older ChatAgent is
supported identically), where predict is an abstract method each agent
implements — so there's no single concrete method on the base to wrap.
install_databricks_monitoring intercepts subclass creation and wraps each
concrete agent's predict as it's defined, composing on top of MLflow's own
validation wrapper:
from otelzero import install_databricks_monitoring
# 1) Install the hook FIRST (it patches ResponsesAgent.__init_subclass__)
install_databricks_monitoring(server_url="http://localhost:8000", user="Sarah")
# 2) Now define your agent — its predict is auto-instrumented at class creation
from mlflow.pyfunc import ResponsesAgent
from mlflow.types.responses import ResponsesAgentRequest, ResponsesAgentResponse
class MyAgent(ResponsesAgent):
def predict(self, request: ResponsesAgentRequest) -> ResponsesAgentResponse:
... # each call streams agent_start/agent_end/agent_error to the dashboard
A complete, runnable version is in demo/demo_databricks.py (fully offline —
its agent answers by calling two local stand-in tools, so no Databricks workspace
or LLM key is needed):
uv pip install mlflow # provides mlflow.pyfunc.ResponsesAgent
uv run python demo/demo_databricks.py
Beyond the agent run, the tools a ResponsesAgent calls — recorded in its
response as function_call items (with results in matching
function_call_output items) — are surfaced as nested task_start/task_end
events, one per tool, on the agent's timeline. So the demo run shows
agent_start → a task per tool → agent_end, with no extra agent code.
A production Agent Bricks agent (Knowledge Assistant, Information Extraction,
Multi-Agent Supervisor, Custom LLM) is the same ResponsesAgent interface and is
instrumented identically. Only predict is wrapped (the required entry point);
predict_stream is left alone to avoid double-counting agents whose predict
aggregates it internally.
Instrument a real Claude Agent SDK agent
The Claude Agent SDK
exposes two async-generator entry points, and install_claude_agent_monitoring
wraps both so a run is captured exactly once whichever style you use: the
one-shot claude_agent_sdk.query(...) function, and the interactive
ClaudeSDKClient.receive_response() method. ClaudeSDKClient.query() is wrapped
too — not as a run, but so the prompt it sends is recorded on the
receive_response() that answers it. (Note: the SDK's own hooks= option
— PreToolUse/PostToolUse — is unrelated to otelzero's import hooks.)
from otelzero import install_claude_agent_monitoring
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
install_claude_agent_monitoring(server_url="http://localhost:8000", user="Sarah")
# 2) Now import the SDK — query and ClaudeSDKClient are auto-instrumented
from claude_agent_sdk import query, ClaudeAgentOptions
options = ClaudeAgentOptions(model="claude-opus-4-8", allowed_tools=["Read"])
# Each query() run streams agent_start/agent_end/agent_error events
async for message in query(prompt="Summarize README.md", options=options):
...
A complete, runnable version is in demo/demo_claude_sdk.py:
uv pip install claude-agent-sdk # the hook instruments the real package
export ANTHROPIC_API_KEY=... # for a full run
uv run python demo/demo_claude_sdk.py
The agent_start details carry the run's model, allowed_tools, and (names
only) mcp_servers. With tracing on (the default), the SDK's own spans nest
underneath otelzero's invoke_agent Claude Agent span.
Order matters: instrumentation must be installed before
claude_agent_sdkis imported. The hooks use thefind_specprotocol and require Python ≥ 3.12.
Instrument a real Microsoft Agent Framework agent
Microsoft Agent Framework (the
agent-framework package that unifies Semantic Kernel and AutoGen) ships
OpenTelemetry instrumentation in the GenAI semantic conventions. Every
Agent.run, streaming or not, gets an invoke_agent {name} span, with chat
and execute_tool spans from its chat client and function-invocation layers.
install_microsoft_monitoring wraps nothing. It makes otelzero's OTLP provider
the global one and calls agent_framework.observability.enable_instrumentation
with enable_sensitive_data following otelzero's content opt-in. It keeps the
framework on the latest conventions even when OTEL_SEMCONV_STABILITY_OPT_IN is
set for another library, stamps otelzero's run identity on every span, and
turns each invoke_agent span into agent_start / agent_end / agent_error:
from otelzero import install_microsoft_monitoring
from agent_framework import Agent # import order does not matter
from agent_framework.openai import OpenAIChatClient
install_microsoft_monitoring(server_url="http://localhost:8000", user="Sarah")
agent = Agent(OpenAIChatClient(model="gpt-4o"), name="Weather Agent")
await agent.run("What's the weather in Amsterdam?") # non-streaming
async for update in agent.run("...", stream=True): # streaming
...
# both paths: the framework's own spans -> registry, agent_* events from them
If the application called disable_instrumentation(), the hook keeps that
choice and says so. With sensitive data off, a failed run's error is the
exception type: that's all Agent Framework records then.
A complete, runnable version is in demo/demo_microsoft.py (fully offline —
it drives the agent with a tiny local echo chat client, so no Azure/OpenAI key is
needed; it exercises both the non-streaming and streaming run paths):
uv pip install agent-framework "otelzero[otel]"
uv run python demo/demo_microsoft.py
Instrument a real Snowflake Cortex Agent SDK agent
Snowflake's Cortex Code Agent SDK
(the cortex_code_agent_sdk package) is shaped almost identically to the Claude
Agent SDK: a module-level query(prompt, options=...) async-generator function
and an interactive CortexCodeSDKClient whose receive_response() is an async
generator, both configured by a CortexCodeAgentOptions dataclass.
install_snowflake_monitoring wraps both run chokepoints so a run is captured
exactly once whichever style you use, and CortexCodeSDKClient.query() so the
prompt it sends is recorded on the receive_response() that answers it:
from otelzero import install_snowflake_monitoring
# 1) Install the hook FIRST (registers a find_spec meta-path finder)
install_snowflake_monitoring(server_url="http://localhost:8000", user="Sarah")
# 2) Now import the SDK — query and CortexCodeSDKClient are auto-instrumented
from cortex_code_agent_sdk import query, CortexCodeAgentOptions
options = CortexCodeAgentOptions(model="claude-sonnet-4-6", cwd=".")
# Each query() run streams agent_start/agent_end/agent_error events
async for message in query(prompt="Summarize this codebase", options=options):
...
A complete, runnable version is in demo/demo_snowflake.py:
uv pip install cortex-code-agent-sdk # the hook instruments the real package
# authenticate the Snowflake CLI connection the SDK uses (e.g. `snow connection add ...`)
uv run python demo/demo_snowflake.py
The agent_start details carry the run's model, allowed_tools, and (names
only) mcp_servers. With tracing on (the default), the SDK's own spans nest
underneath otelzero's invoke_agent Snowflake Cortex Agent span.
Order matters: instrumentation must be installed before
cortex_code_agent_sdkis imported. The hooks use thefind_specprotocol and require Python ≥ 3.12.
OpenTelemetry Tracing
otelzero can emit standard OpenTelemetry traces alongside (or instead of) the built-in telemetry. The same import hooks that stream dashboard events also open spans, so a crew run becomes a real distributed trace:
Crew.kickoff→ aninvoke_workflowspan, with eachAgent.execute_taskandTask.execute_syncas nested child spans (real parent/child via context).- An error sets the span status to
ERROR, records the exception, and setserror.type. - Attributes follow the GenAI semantic conventions — see OTel span attributes.
It's on by default and dependency-light: install the otel extra and the
spans flow (OTELZERO_OTEL is effectively always 1). Without the extra, the
client runs exactly as before, with zero OpenTelemetry deps. OTELZERO_OTEL=0
switches the import hooks' spans off; the native-OTel hooks (Pydantic AI, Strands,
ADK, Microsoft, LangChain / LangGraph) keep tracing, because spans are how they
report.
uv sync --extra otel # opentelemetry-api / sdk / exporter-otlp
uv run python demo/demo_crewai.py
By default spans are exported to the registry's own OTLP receiver at POST /v1/traces,
which translates them back into the dashboard's logs and stats — so the existing
UI keeps working. To send traces to a standard backend instead (Jaeger, Tempo,
Grafana, an OpenTelemetry Collector), point the exporter elsewhere with the
standard env var:
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
uv run python demo/demo_crewai.py
Dual-emit note: with tracing on (the default), each run reaches Kiji's own server both as events (
/api/telemetry) and as spans (/v1/traces). If your registry counts both, pointOTEL_EXPORTER_OTLP_ENDPOINTat a separate collector, or setOTELZERO_OTEL=0for the import-hook frameworks.
OpenTelemetry tracing requires Python ≥ 3.12 (as do the import hooks).
How LangSmith's spans are translated
install_langchain_monitoring / install_langgraph_monitoring switch on
LangChain's own OpenTelemetry connector (the one that ships with the langsmith
SDK; see the LangGraph guide).
The connector's spans carry a langsmith.span.kind run type, but they speak an
older dialect of the GenAI conventions: gen_ai.operation.name holds the
run type (chain, parser) and labels prompt-template steps chat, the
deprecated gen_ai.system holds langchain, and the prompt and response leave
the process as gen_ai.prompt / gen_ai.completion whether or not you opted
in. The one piece of otelzero in the pipeline fixes that: wrap the OTLP exporter
in otelzero.translate.TranslatingExporter and every span is rewritten into the
current conventions on its way out —
| LangChain's connector emits | Translated to |
|---|---|
root chain run (langsmith.trace.name = run_name) |
invoke_agent {run_name}, gen_ai.agent.name |
langsmith.span.kind = llm |
chat {model}, gen_ai.provider.name from ls_provider, gen_ai.request.model |
langsmith.span.kind = tool |
execute_tool {tool}, gen_ai.tool.name |
nested chain / prompt / parser steps |
no gen_ai.operation.name — they are not GenAI operations |
gen_ai.system, gen_ai.serialized.*, LANGSMITH_* metadata |
dropped |
gen_ai.prompt / gen_ai.completion |
gen_ai.input.messages / gen_ai.output.messages / gen_ai.system_instructions — only with OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1; dropped otherwise |
gen_ai.usage.total_tokens |
otelzero.usage.total_tokens (input/output tokens keep their names) |
run metadata conversation_id / thread_id, user_name / user_id |
gen_ai.conversation.id, user.name |
status OK on success |
left unset, as the conventions require; error.type filled in on failure |
The mapping lives in otelzero.translate.langsmith.translate_attributes, a pure
function over plain attribute dicts, so an OTLP backend can run the identical
mapping on ingest. otel.init_tracing installs the exporter wrapper on
otelzero's own provider too, so an otelzero user who also turns the connector on
gets translated spans without further setup.
Because the connector builds its spans on LangSmith's background tracing
thread, a run_context is not visible to them. A LangChain run names its user
and conversation through its config instead (LangGraph's
config={"configurable": {"thread_id": ...}} works too):
chain.invoke(
{"input": query},
config={"run_name": "SQL Agent",
"metadata": {"user_name": "Sarah", "conversation_id": thread_id}},
)
Kiji maps the translated spans straight onto the dashboard:
- each
invoke_agentrun → an agent card (tag the run withrun_nameto label it, e.g.ReAct Agentinstead ofRunnableSequence); - nested
chat/execute_toolspans → task rows in the live feed, labeled by span name — without spawning their own cards; - a failing run sets the span status to
ERROR→ a failed run on the card.
Ingest Microsoft Copilot Studio (SaaS) by relaying its OTLP telemetry
Copilot Studio is a
hosted, low-code product — there is no Python process to import or method to
wrap, so the import-hook path does not apply. Instead, Copilot Studio agents
export telemetry to Azure Monitor / Application Insights following the OTel
GenAI semantic conventions — the same invoke_agent / chat /
execute_tool spans Kiji's /v1/traces receiver already maps. So ingesting
Copilot Studio is a matter of relaying those spans to the registry's OTLP endpoint —
no new otelzero code:
┌──────────────────┐ GenAI/OTel ┌───────────────┐ OTLP ┌───────────────┐
│ Copilot Studio │─────────────→│ OTel │──────────→│ Kiji │
│ (Azure Monitor) │ telemetry │ Collector │ /v1/traces│ dashboard │
└──────────────────┘ └───────────────┘ └───────────────┘
The final Collector hop is a small, reusable config —
demo/otelcol-copilot-studio.yaml — an OTLP
receiver → otlphttp exporter to …/v1/traces with the ingest key, stamping
service.name: "Copilot Studio" so runs group under one framework label:
OTELZERO_API_KEY=<ingest-key> \
otelcol-contrib --config demo/otelcol-copilot-studio.yaml
How spans reach the Collector depends on your Azure setup:
- Pro-code / Foundry-backed agents that let you set an OTLP endpoint: point
OTEL_EXPORTER_OTLP_ENDPOINTat the Collector — spans arrive directly, no Azure round-trip. - Pure Copilot Studio SaaS agents export only to Application Insights. Relay those by fanning App Insights out via a diagnostic-settings export to Event Hub (or a scheduled Log Analytics / KQL query) and a small converter that re-emits the GenAI records as OTLP into the Collector. (The Collector has an Azure Monitor exporter, not a receiver, so the converter step is required for the SaaS-only path.)
To validate the Kiji side end to end without an Azure tenant,
demo/demo_copilot_studio_otel.py synthesizes the exact OTLP spans a Copilot
Studio agent produces and posts them to /v1/traces over real OTLP/HTTP — just
as the relaying Collector would:
(cd ../kiji-agent-registry && uv run kiji-agent-registry) # terminal 1
uv run python demo/demo_copilot_studio_otel.py # terminal 2 (no Azure needed)
Each invoke_agent run becomes an agent card; nested chat / execute_tool
spans become labeled task rows; an errored tool propagates to a failed run —
identical rendering to the framework integrations above.
How It Works
Import Hook Magic
The instrumentation uses Python's import system to intercept module loading.
Python 3.12 removed the legacy find_module protocol, so the hook implements
find_spec and wraps the real loader to instrument the module after it
executes:
class CrewAIImportHook(importlib.abc.MetaPathFinder):
def find_spec(self, fullname, path=None, target=None):
if fullname not in ("crewai.crew", "crewai.agent", "crewai.task"):
return None
# Resolve the real spec from the rest of sys.meta_path (guarded against
# re-entrancy), then swap in a loader that instruments after exec.
spec = importlib.util.find_spec(fullname)
spec.loader = _InstrumentingLoader(spec.loader, self, fullname)
return spec
class _InstrumentingLoader(importlib.abc.Loader):
def exec_module(self, module):
self._real_loader.exec_module(module) # load CrewAI normally
self._hook._instrument_module(module) # then wrap Crew/Agent/Task
Function Wrapping
Uses wrapt library for clean function wrapping:
@wrapt.decorator
def agent_wrapper(wrapped, instance, args, kwargs):
# Before execution
send_log("agent_start", agent_role=instance.role)
try:
result = wrapped(*args, **kwargs)
# On success
send_log("agent_end", agent_role=instance.role)
return result
except Exception as e:
# On error
send_log("agent_error", error=str(e), traceback=...)
raise
Async Telemetry
Logs are sent asynchronously to avoid blocking agent execution:
def send_log(self, event_type, **kwargs):
log_data = {"event_type": event_type, ...}
# Send in background thread
threading.Thread(
target=self._send_async,
args=(log_data,),
daemon=True
).start()
Event Types
crew_start- Crew execution beginscrew_end- Crew execution completescrew_error- Crew execution failsagent_start- Agent begins task executionagent_end- Agent completes taskagent_error- Agent encounters errortask_start- Task execution beginstask_end- Task execution completestask_error- Task execution fails
Common telemetry envelope
Every event (send_log) carries the same top-level fields regardless of
framework: event_type, timestamp, agent_name, agent_role, framework,
framework_version, model, user, details, error, traceback — plus a
parallel OpenTelemetry span (tracing is on by default).
Per-framework instrumentation
service.name carries the framework label; the span names below follow the
GenAI conventions' {operation} {subject} templates.
service.name is the framework label the client is created with, and it is also
the value the dashboard groups and filters runs by. Set OTEL_SERVICE_NAME, or
service.name in OTEL_RESOURCE_ATTRIBUTES, to replace it.
| Framework | service.name |
Wrapped method(s) | OTel span(s) | Lifecycle events |
|---|---|---|---|---|
| CrewAI | CrewAI |
Crew.kickoff, Agent.execute_task, Task.execute_sync |
invoke_workflow {crew}, invoke_agent {role}, execute_task {role} |
crew + agent + task |
| LangChain | LangChain |
— (LangSmith's native OTel export, translated) | invoke_agent {run} → chat {model}, execute_tool {tool}, plus one span per nested chain / prompt / parser step (LangSmith's own) |
agent |
| LangGraph | LangGraph |
— (LangSmith's native OTel export, translated) | invoke_agent {graph} → {node} → chat {model}, execute_tool {tool} (LangSmith's own) |
agent |
| LlamaIndex | LlamaIndex |
Workflow.run (every agent, and every workflow) |
invoke_agent {name} |
agent |
| OpenAI Agents | OpenAI Agents |
Runner.run, run_sync, run_streamed |
invoke_agent {name} |
agent |
| Pydantic AI | Pydantic AI |
— (native OTel via Agent.instrument_all) |
invoke_agent {name} → chat, execute_tool (Pydantic AI's own) |
agent |
| smolagents | smolagents |
MultiStepAgent.run (direct + stream) |
invoke_agent {name} |
agent |
| AWS Strands | Strands Agents |
— (native OTel, latest GenAI conventions) | invoke_agent {name} → execute_event_loop_cycle, chat, execute_tool (Strands' own) |
agent |
| Google ADK | Google ADK |
— (native OTel, latest GenAI conventions) | invoke_workflow → invoke_agent {name} → generate_content, execute_tool (ADK's own) |
agent |
| Databricks | Databricks Agents |
ResponsesAgent.predict, ChatAgent.predict |
invoke_agent {name}, execute_tool {tool} |
agent + task |
| Claude Agent SDK | Claude Agent SDK |
query, ClaudeSDKClient.receive_response |
invoke_agent Claude Agent |
agent |
| Microsoft | Microsoft Agent Framework |
— (native OTel via enable_instrumentation) |
invoke_agent {name} → chat, execute_tool (Agent Framework's own) |
agent |
| Snowflake Cortex | Snowflake Cortex |
query, CortexCodeSDKClient.receive_response |
invoke_agent Snowflake Cortex Agent |
agent |
Event matrix — which lifecycle events each framework emits
| Framework | crew_* |
agent_* |
task_* |
|---|---|---|---|
| CrewAI | ✅ | ✅ | ✅ |
| LangChain | — | ✅⁷ | — |
| LangGraph | — | ✅⁷ | — |
| LlamaIndex | — | ✅ | —⁴ |
| OpenAI Agents | — | ✅ | — |
| Pydantic AI | — | ✅⁵ | — |
| smolagents | — | ✅ | — |
| Strands | — | ✅⁵ | — |
| Google ADK | — | ✅⁵ | — |
| Databricks | — | ✅ | ✅² |
| Claude Agent SDK | — | ✅ | — |
| Microsoft | — | ✅⁵ | —³ |
| Snowflake Cortex | — | ✅ | — |
² Databricks maps each ResponsesAgent tool call (a function_call item in the
response) to a task_* pair, nested under the agent run.
³ Microsoft's nested model/tool calls surface as task_* rows when its
native OTel spans are ingested (invoke_agent → agent, chat/execute_tool
→ tasks), which install_microsoft_monitoring sets up.
⁴ Likewise for LlamaIndex: the hook captures the agent run, while its nested
LLM/tool/retriever steps surface as task_* rows when ingested via
OpenInference (openinference-instrumentation-llama-index; root
CHAIN/AGENT → agent, everything nested → tasks), which Kiji's receiver also
ingests.
⁷ LangChain / LangGraph agent_* events come from LangChain's callbacks, one
pair per top-level run; every nested chain, node, model and tool call is a span
from LangSmith's export, not an event.
⁵ The native-OTel frameworks' (Pydantic AI, Strands, ADK, Microsoft) agent_*
events come from their own invoke_agent spans, not from a wrapper; their model
and tool spans reach the registry over OTLP only.
Details captured per wrapped call
| Framework · method | *_start details |
*_end details |
*_error |
|---|---|---|---|
CrewAI · Crew.kickoff |
num_agents, num_tasks |
status: completed |
error, traceback |
CrewAI · Agent.execute_task |
role, goal (≤100 chars) |
status: success |
error, traceback |
CrewAI · Task.execute_sync |
description, expected_output (≤100) |
status: completed |
error, traceback |
| LangChain / LangGraph · top-level run (LangChain callbacks) | model when the run is a model call |
status: success, model, tools (called), usage |
error, traceback |
LlamaIndex · Workflow.run |
model, tools (+ agents for an AgentWorkflow) |
status: success |
error, traceback |
OpenAI Agents · Runner.run / run_sync / run_streamed |
model, tools, handoffs |
status: success |
error, traceback |
Pydantic AI · native invoke_agent span |
model |
status: success, tools, usage |
error, traceback⁶ |
smolagents · MultiStepAgent.run |
model, tools |
status: success |
error, traceback |
Strands · native invoke_agent span |
model (none: set after start) |
status: success, model, tools, usage |
error, traceback |
Google ADK · native invoke_agent span |
model (none: set after start) |
status: success, model, tools, usage (summed over its generate_content spans) |
error, traceback |
Databricks · ResponsesAgent.predict / ChatAgent.predict |
num_messages |
status: success |
error, traceback |
Databricks · tool call (ResponsesAgent) |
tool, call_id, arguments (≤200) |
status: success, output (≤200) |
error, traceback |
Claude Agent SDK · query / ClaudeSDKClient.receive_response |
model, allowed_tools, mcp_servers (names) |
status: success |
error, traceback |
Microsoft · native invoke_agent span |
model (none: set after start) |
status: success, model (from its chat spans), tools, usage |
error, traceback⁶ |
Snowflake Cortex · query / CortexCodeSDKClient.receive_response |
model, allowed_tools, mcp_servers (names) |
status: success |
error, traceback |
⁶ Pydantic AI and Agent Framework record the exception message and traceback
only when content capture is on; otherwise error is the exception type and
traceback is empty.
Prompt and response (opt-in). With
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1 set on the agent, every
*_end / *_error row above also carries input_messages (the run's prompt)
and output_messages (its response), plus system_instructions where the hook
can see a system prompt. Both are arrays in the GenAI conventions' message shape
({"role", "parts": [{"type": "text", "content"} | reasoning | tool_call | tool_call_response]});
the model's stop reason sits beside them as finish_reasons, not inside a
message — the same content that goes
out as the gen_ai.input.messages / gen_ai.output.messages span attributes,
so the dashboard, the apps and an OTLP backend all see one record. Off by
default: no prompt or response text leaves the process unless you opt in. Size
is bounded on the client: OTELZERO_CONTENT_MAX_CHARS (default 4000, 0
disables) caps each text / arguments / response part with a
…[truncated, N more chars] marker, and OTELZERO_CONTENT_MAX_MESSAGES
(default 20, 0 disables) keeps only the most recent input messages of a
replayed chat history.
OTel span attributes
otelzero follows the OpenTelemetry GenAI semantic
conventions
(status: development). The vocabulary has a single definition in
src/otelzero/semconv.py, mirrored in the registry's
kiji_agent_registry/semconv.py (a registry test checks the two agree).
On every span:
| Attribute | Value |
|---|---|
gen_ai.operation.name |
invoke_workflow, invoke_agent, execute_tool, or execute_task |
gen_ai.conversation.id |
the run's flow id — groups a conversation's events |
user.name |
who the run is for |
otelzero.session.name |
the session's title, once it has one (name= or derived from the run's intent) |
error.type |
the exception class name, on failed spans only |
A successful span carries no status code: the conventions require it to be
left unset, so only failures set ERROR (plus error.type and the exception
message).
Every otelzero span is SpanKind.INTERNAL, and the conventions define a smaller
attribute set for the internal invoke_agent span than for the client one.
Two attributes you might expect are therefore absent by design:
gen_ai.provider.name. It replaced the deprecatedgen_ai.system, but it names the inference provider — which an orchestrator like CrewAI is not — and it is listed only on thecreate_agentandinvoke_agentclient spans. The framework (crewai,langchain, …) rides on theservice.nameresource attribute instead.gen_ai.agent.id. Also client-span-only, and the spec calls recording in-memory agent instance ids on it NOT RECOMMENDED "due to their transient nature" — which is all Kiji's hooks can see.
This applies to the spans otelzero emits itself. The native-OTel frameworks
(Pydantic AI, Strands, ADK, Microsoft) emit their own spans, which carry
whatever the framework records: Strands and Agent Framework set
gen_ai.provider.name (strands-agents, microsoft.agent_framework), and
Agent Framework sets gen_ai.agent.id.
Kiji's ingest still reads both from third-party traces that legitimately set them.
Token usage, on the span that closes a run:
| Attribute | Meaning |
|---|---|
gen_ai.usage.input_tokens / gen_ai.usage.output_tokens |
prompt / completion tokens |
gen_ai.usage.reasoning.output_tokens |
reasoning tokens, when the provider reports them |
gen_ai.usage.cache_read.input_tokens |
tokens served from a provider cache |
otelzero.usage.total_tokens |
otelzero's own — the conventions record input and output separately and define no total |
Finish reasons, on the span that closes a run, whether or not content
capture is on: gen_ai.response.finish_reasons, the conventions' array of
normalised stop reasons (stop, length, tool_call, content_filter, …),
one per generation the framework reports — every turn of a streamed run, the
final one of a one-shot result.
It replaces the per-message finish_reason field the conventions deprecated,
and rides on the log row as finish_reasons.
Content (Opt-In), on the span that closes a run, only when
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1:
| Attribute | Meaning |
|---|---|
gen_ai.input.messages |
the run's prompt, as a JSON array of {"role", "parts"} messages |
gen_ai.output.messages |
the run's response, same shape (one message per turn) |
gen_ai.system_instructions |
the system prompt as a JSON array of parts (CrewAI agents: the backstory) |
What counts as the prompt and the response per framework:
| Framework | Prompt (gen_ai.input.messages) |
Response (gen_ai.output.messages) |
|---|---|---|
| CrewAI | kickoff(inputs=) as JSON; a task's description (+ context); an agent's executor history up to its prompt (its system prompt as system_instructions) |
CrewOutput.raw / TaskOutput.raw; an agent's whole exchange (tool calls, results, answer) from its executor history |
| LangChain / LangGraph | exported by LangSmith itself as gen_ai.prompt (hidden — {} — unless content capture is on) |
likewise gen_ai.completion |
| Strands | recorded by Strands itself (latest conventions, gen_ai_span_attributes_only) |
likewise |
| Google ADK | recorded by ADK itself on its generate_content spans (with SPAN_ONLY / SPAN_AND_EVENT) |
likewise |
| Databricks | ChatAgent messages, or ResponsesAgent request.input items (message, function_call, function_call_output) |
response.messages or response.output items |
| Claude Agent SDK | the query(prompt) string, or a streamed prompt's messages (teed as the SDK reads them); for the client, what client.query() sent, one prompt per receive_response() |
every streamed AssistantMessage/UserMessage block (text, tool use, tool result), else ResultMessage.result; stop_reason normalised |
| Microsoft | recorded by Agent Framework itself on its invoke_agent / chat spans (sensitive data on) |
likewise |
| Snowflake Cortex | as Claude Agent SDK | as Claude Agent SDK |
Tool calls inside a message map to {"type": "tool_call", "id", "name", "arguments"}
and their results to {"type": "tool_call_response", "id", "response"}, filed
under the tool role; Anthropic tool_use/tool_result, OpenAI
function_call/function_call_output, LangChain tool_calls and Gemini
function_call/function_response blocks are all recognised. Content is
capped on the client (OTELZERO_CONTENT_MAX_CHARS, OTELZERO_CONTENT_MAX_MESSAGES,
see Details captured per wrapped call).
The registry's ingest stores these attributes as the same input_messages /
output_messages details the HTTP path records, and its export writes them back
as gen_ai.* when the server process has opted in — otherwise as
otelzero.input_messages / otelzero.output_messages (still JSON), which
ingest also reads, so a downloaded trace re-ingests losslessly either way.
Per span, beyond the common set (every span below also carries the Opt-In content attributes † when captured):
| Span | Operation | Extra attributes |
|---|---|---|
invoke_workflow {crew} (CrewAI) |
invoke_workflow |
gen_ai.workflow.name, gen_ai.request.model, otelzero.num_agents, otelzero.num_tasks |
invoke_agent {role} (CrewAI) |
invoke_agent |
gen_ai.agent.name (role), gen_ai.agent.description (goal), gen_ai.request.model |
execute_task {role} (CrewAI) |
execute_task |
gen_ai.agent.name, otelzero.task.description, otelzero.task.expected_output |
invoke_agent {name} (LlamaIndex) |
invoke_agent |
gen_ai.agent.name, gen_ai.request.model ‡, gen_ai.tool.definitions † |
invoke_agent {name} (OpenAI Agents) |
invoke_agent |
gen_ai.agent.name, gen_ai.request.model, gen_ai.tool.definitions † |
invoke_agent {name} (Pydantic AI, emitted by Pydantic AI itself) |
invoke_agent |
gen_ai.agent.name, gen_ai.aggregated_usage.*; child chat spans carry gen_ai.request.model, gen_ai.usage.*, gen_ai.tool.definitions |
invoke_agent {name} (smolagents) |
invoke_agent |
gen_ai.agent.name, gen_ai.request.model, gen_ai.tool.definitions † |
invoke_agent {name} (Strands, emitted by Strands itself) |
invoke_agent |
gen_ai.agent.name, gen_ai.provider.name, gen_ai.request.model, gen_ai.agent.tools, gen_ai.usage.*, gen_ai.tool.definitions † |
invoke_agent {name} (ADK, emitted by ADK itself) |
invoke_agent |
gen_ai.agent.name, gen_ai.conversation.id (ADK's session); child generate_content spans carry gen_ai.request.model, gen_ai.usage.*, gen_ai.tool.definitions |
invoke_agent {name} (Databricks) |
invoke_agent |
gen_ai.agent.name, otelzero.num_messages |
execute_tool {tool} (Databricks) |
execute_tool |
gen_ai.agent.name, gen_ai.tool.name, gen_ai.tool.call.id |
invoke_agent Claude Agent |
invoke_agent |
gen_ai.agent.name, gen_ai.request.model, gen_ai.tool.definitions † |
invoke_agent {name} (Microsoft, emitted by Agent Framework itself) |
invoke_agent |
gen_ai.agent.name, gen_ai.agent.id, gen_ai.provider.name, gen_ai.tool.definitions, gen_ai.usage.*; child chat spans carry gen_ai.request.model |
invoke_agent Snowflake Cortex Agent |
invoke_agent |
gen_ai.agent.name, gen_ai.request.model, gen_ai.tool.definitions † |
Span names follow the conventions' {operation} {subject} templates and fall
back to the bare operation when the subject isn't available (an unnamed crew
becomes just invoke_workflow). execute_task is a documented custom operation
value: a CrewAI Task is the parent of the agent invocation it delegates to, so
no predefined value fits, and the enum permits a custom one where none applies.
‡ LlamaIndex: whenever the agent's LLM does not name itself (its
model/model_name is absent or the literal "unknown"): the model is then
taken from what the provider reported on the first reply, falling back to the LLM
class name. An LLM that does name its model reports it from the start.
† The content attributes — gen_ai.tool.definitions, gen_ai.input.messages,
gen_ai.output.messages, gen_ai.system_instructions — are Opt-In in the
conventions, so they appear only when you set
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=1. The tool names also
ride the log envelope's details regardless (they are not content), so Kiji's
dashboard and apps list them either way; the prompt and response ride details
only under the same opt-in.
Deliberate deviations
Two places where otelzero knowingly emits more than a span's definition lists. Extra attributes are permitted; these are the ones worth knowing about:
gen_ai.usage.*andgen_ai.request.modeloninvoke_workflowspans. That span defines onlyoperation.name,error.type,conversation.id, andworkflow.name. CrewAI reports token usage only at the crew level, so honouring the list literally would throw the numbers away.gen_ai.conversation.idanduser.nameon every span, includingexecute_tool, whereconversation.idisn't listed. Both are load-bearing for keeping concurrent users' flows apart.
Not yet implemented, for reference: gen_ai.usage.cache_creation.input_tokens,
the request parameters (temperature, max_tokens, top_p, …),
gen_ai.tool.description / gen_ai.tool.type, and
the Opt-In execute_tool content attributes (gen_ai.tool.call.arguments,
gen_ai.tool.call.result) — tool arguments and results are recorded today only
as parts inside gen_ai.input.messages / gen_ai.output.messages.
Splitting flows per user
TelemetryClient's user and session are process-wide, which is right for a
script and wrong for a hosted agent — many callers in one process would report
the same user and collapse into one flow. Wrap each run to give it its own
identity:
import otelzero
with otelzero.run_context(user="alice", conversation_id=thread_id):
crew.kickoff(...)
user and conversation_id become the user.name and gen_ai.conversation.id
span attributes and the log envelope's user / session. Omit
conversation_id for a fresh flow per run, or pass your own thread id to keep a
multi-turn conversation together. The scope lives in a ContextVar, so
concurrent asyncio tasks and threads each see their own.
Migrating from the pre-1.0 names
The client emits only the names above. The server reads them and their pre-1.0 spellings, so a client older than the server it reports to keeps producing agent cards and token counts unchanged.
| Pre-1.0 | Now |
|---|---|
gen_ai.system |
removed — framework is the service.name resource attribute |
gen_ai.operation.name: crew / agent / task |
invoke_workflow / invoke_agent / execute_task |
mira.user (resource) |
user.name (span) |
| (session was the trace id) | gen_ai.conversation.id (span) |
mira.usage.reasoning_tokens |
gen_ai.usage.reasoning.output_tokens |
mira.usage.cache_read_tokens |
gen_ai.usage.cache_read.input_tokens |
mira.agent.goal |
gen_ai.agent.description |
mira.tool.call_id |
gen_ai.tool.call.id |
mira.agent.tools |
gen_ai.tool.definitions (opt-in — see above) |
gen_ai.agent.id |
otelzero.agent.id — client-span-only, and the values are transient instance ids |
(span status was OK on success) |
left unset, as the conventions require |
mira.chain, mira.app_name |
gen_ai.workflow.name |
mira.usage.total_tokens, mira.num_agents, mira.num_tasks, mira.num_messages, mira.steps, mira.agent.class, mira.task.* |
the same attributes, renamed to otelzero.* — no convention equivalent |
Third-party emitters you don't control (LangChain's LangSmith connector,
Azure Monitor's Copilot Studio export) are unaffected: setting mira.user on
the resource still attributes their traces.
CrewAI and Databricks both surface a task level nested under the agent — CrewAI
because its framework exposes task execution as a wrappable method, Databricks by
reading the tool calls out of each ResponsesAgent response. The OpenAI Agents
SDK, smolagents, the Claude Agent SDK and the Snowflake Cortex Agent SDK are
instrumented at a single agent-run boundary. Pydantic AI, Strands, ADK, Microsoft,
LangChain and LangGraph are not wrapped at all: their own native OTel spans
(agent run, model and tool calls) are the trace.
Customization
Change Server URL
install_crewai_monitoring(server_url="http://your-server:8000")
Add Custom Instrumentation
Edit _instrument_module() in the import hook:
def _instrument_module(self, fullname, module):
if fullname == 'your_custom_module':
self._wrap_your_class(module)
Troubleshooting
Client can't connect to server
- Ensure the registry is running (
uv run kiji-agent-registryin its checkout) - Check firewall settings
- Verify server URL in client code
No logs appearing
- Check console for error messages
- Verify instrumentation installed before CrewAI import
- Check network connectivity
Dashboard not updating
- Verify JavaScript console for errors
- Check browser compatibility (modern browsers only)
- Ensure server is returning data at /api/activity
Development
uv sync --extra otel
uv run pytest
To release to PyPI, create a release/x.y.z branch, set the version to x.y.z
in pyproject.toml (under both [project] and [tool.commitizen]) and
src/otelzero/__init__.py, run uv lock, update CHANGELOG.md, and open a pull
request into main. Merging it runs the tests and publishes that version to PyPI. Every push
to main also publishes a dev build to TestPyPI.
License
Apache-2.0
Metadata
Release files for otelzero 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| otelzero-0.2.0.tar.gz | 203.3 kB | With provenance | Details for otelzero-0.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| otelzero-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for otelzero-0.2.0-py3-none-any.whl |
Total release size: 339.0 kB
Release files / otelzero-0.2.0.tar.gz
| Download URL | otelzero-0.2.0.tar.gz |
|---|---|
| Size | 203.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
664aecf35a30dfb0dbcb46a74baf9c2e29534075be341ff47e94ed64a29d6796
|
|
BLAKE2b-256 checksum How to use checksums |
fd0f89d19863ee81df5b3f23a636758b607db2e7c878ae7d0f7d9476882e5e92
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.
Transparency logRelease files / otelzero-0.2.0-py3-none-any.whl
| Download URL | otelzero-0.2.0-py3-none-any.whl |
|---|---|
| Size | 135.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5fb930f170f0ff80d1e0b254152f8aa5d374e44123bc6c7766db24913eac7414
|
|
BLAKE2b-256 checksum How to use checksums |
ab0e6b259125d689c195a30ac5a72cf4a9c788b454e3eedc3e65c45e0b92fbb4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.
Transparency logSecurity for otelzero 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for otelzero-0.2.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.