agentmetrics-llamaindex

AgentMetrics integration for LlamaIndex. Call instrument() once and every agent run and query engine response reports back to your dashboard showing latency, cost, token counts, tool calls, and errors, via LlamaIndex's native instrumentation API with zero changes to your agent code.

Install

pip install agentmetrics-llamaindex

Quickstart

from agentmetrics_llamaindex import instrument

# Register on the global root dispatcher once at startup
span_handler = instrument(
    agent_id="my-llamaindex-agent",
    base_url="http://localhost:8099",
)

# Run your agents and query engines as normal
response = agent.chat("Summarize this document")

span_handler.flush()

API

instrument(agent_id, base_url)

Registers AgentMetricsSpanHandler and AgentMetricsEventHandler on the global LlamaIndex root dispatcher. Returns the span handler for flushing.

AgentMetricsSpanHandler

LlamaIndex BaseSpanHandler that tracks top-level agent/engine spans. Emits a run summary on span completion or error.

AgentMetricsEventHandler

LlamaIndex BaseEventHandler that accumulates token counts and tool calls from LLMChatEndEvent, LLMCompletionEndEvent, and AgentToolCallEvent.

.flush(timeout=10.0)

Blocks until all in-flight HTTP requests complete.

What gets tracked

Each top-level agent or query engine span emits one event to /v1/events:

The handler detects top-level spans by checking whether the span has no parent and whether the owning instance is an agent, engine, runner, or query object.

License

MIT