AgentTelemetry

OpenTelemetry-based observability for AI agent systems. Provides 9 agent-specific span kinds, 7 framework adapters, privacy controls, and analysis tools for fault detection in autonomous agents.

Installation

pip install agenttelemetry

Quickstart

from agenttelemetry import configure, start_agent_span, AgentSpanKind

# One-line setup: configure tracing with console output
provider = configure(service_name="my-agent", console=True)
tracer = provider.get_tracer()

# Trace an agent task
with start_agent_span("research-task", AgentSpanKind.AGENT, tracer=tracer) as span:
    span.set_attribute("agent.name", "researcher")

    with start_agent_span("call-gpt4", AgentSpanKind.LLM_CALL, tracer=tracer) as llm:
        llm.set_attribute("llm.model", "gpt-4o")
        llm.set_attribute("llm.input_tokens", 500)
        llm.set_attribute("llm.output_tokens", 200)

    with start_agent_span("web-search", AgentSpanKind.TOOL_CALL, tracer=tracer) as tool:
        tool.set_attribute("tool.name", "search")
        tool.set_attribute("tool.status", "success")

Features

• 9 agent-specific span kinds -- AGENT, LLM_CALL, TOOL_CALL, PLANNING, REASONING, RETRIEVAL, GUARD_RAIL, DELEGATION, MEMORY
• 7 framework adapters -- auto-instrumentation for LangChain, CrewAI, AutoGen, Anthropic SDK, OpenAI SDK, LlamaIndex, and a manual API for custom agents
• 3 privacy levels -- NONE (structural only), METADATA_ONLY (default; adds model names, token counts, costs), FULL (captures prompt/completion content)
• 4 analysis modules -- anomaly detection, cost aggregation, decision attribution, hallucination tracing
• Built on OpenTelemetry -- exports to Jaeger, Grafana Tempo, Datadog, or any OTLP-compatible backend
• Cost estimation -- automatic USD cost calculation for OpenAI and Anthropic models

Framework Adapters

Each adapter uses lazy imports -- the framework package is only required at instrument() time, not at import time.

Auto-instrumentation Example (OpenAI)

from agenttelemetry import configure
from agenttelemetry.adapters import OpenAIInstrumentor

provider = configure(service_name="my-agent", console=True)
instrumentor = OpenAIInstrumentor()
instrumentor.instrument(tracer_provider=provider.tracer_provider)

# All openai.chat.completions.create() calls are now traced automatically

Analysis Modules

Benchmark Results

In fault-detection experiments, AgentTelemetry detects 8 out of 8 fault types at FDR = 1.0:

• Circular delegation loops
• Infinite tool retry loops
• Cost explosion (runaway spending)
• Context window overflow
• Ungrounded LLM outputs (hallucination candidates)
• Unattributed tool decisions
• Token budget violations
• Latency anomalies

Privacy Controls

from agenttelemetry import configure, PrivacyLevel

# Default: captures metadata but not prompt/completion content
provider = configure(service_name="my-agent", privacy_level=PrivacyLevel.METADATA_ONLY)

# Full capture for debugging (opt-in)
provider = configure(service_name="my-agent", privacy_level=PrivacyLevel.FULL)

# Minimal: structural info only (span kinds, timing, status)
provider = configure(service_name="my-agent", privacy_level=PrivacyLevel.NONE)

API Reference

Core

• configure(service_name, privacy_level, console, json_file) -- one-line setup, returns AgentTelemetryProvider
• AgentTelemetryProvider -- wraps OTel TracerProvider with agent-specific defaults
• start_agent_span(name, kind, tracer, attributes) -- context manager for agent spans
• AgentSpanKind -- enum of 9 span kinds
• PrivacyLevel -- NONE, METADATA_ONLY, FULL

Adapters

• AnthropicInstrumentor, OpenAIInstrumentor, LangChainInstrumentor, CrewAIInstrumentor, AutoGenInstrumentor, LlamaIndexInstrumentor
• Manual helpers: start_llm_span, start_tool_span, start_retrieval_span, start_planning_span, start_reasoning_span, start_guardrail_span, start_delegation_span, start_memory_span

Analysis

• AnomalyDetector / Anomaly / AnomalyType
• CostAggregator / CostReport / ModelCost
• DecisionAttributor / ToolDecision
• HallucinationTracer / HallucinationCandidate

Contributing

See CONTRIBUTING.md for development setup, code structure, and guidelines.

License

This project is licensed under the Apache License 2.0. See LICENSE for the full text.

Citation

If you use AgentTelemetry in your research, please cite:

@software{agenttelemetry2025,
  title     = {AgentTelemetry: OpenTelemetry-Based Observability for AI Agent Systems},
  author    = {Balusu, Krishna Chaitanya},
  year      = {2025},
  url       = {https://github.com/agenttelemetry/agenttelemetry},
  license   = {Apache-2.0},
}