llm-agent-trace

Zero-dependency LLM call tracer. Patch once — every provider is captured automatically.

┌─ research_agent · #a3f2b1  1149 tokens  $0.0022  3.4s
│
│  ├─ summarize  312→87 tok  $0.0004  1.2s
│  │  └─► openai · api.openai.com/v1/chat/completions  gpt-4o  312→87 tok  $0.0004  1.2s  ✓
│  │
│  └─ verify  540→210 tok  $0.0018  2.1s
│     └─► anthropic · api.anthropic.com/v1/messages  claude-sonnet-4-6  540→210 tok  $0.0018  2.1s  ✓
│
│  2 call(s)  ·  852→297 tokens  ·  $0.0022  ·  3.4s
└────────────────────────────────────────

Why

Every LLM observability tool is either a paid SaaS, a massive framework dependency, or tied to a specific provider. llm-agent-trace is none of those — it's a small library you drop in and forget about.

• Zero runtime dependencies — stdlib only
• Provider-agnostic — works with OpenAI, Anthropic, Mistral, Groq, Gemini, Cohere, Ollama, LM Studio, Azure OpenAI, and anything that speaks HTTP
• Works with LangChain, LiteLLM, and any other framework — intercepts at the HTTP layer, not the SDK layer
• Sync + async — both are captured

Install

pip install llm-agent-trace

Quickstart

import llm_agent_trace

# One line at the top of your script
llm_agent_trace.patch()

# Wrap your agent run in a session
with llm_agent_trace.session("my_agent"):
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello"}]
    )
# Trace is printed to stderr automatically when the session ends

Works identically with LiteLLM, LangChain, Anthropic SDK, or any HTTP-based LLM client — no changes to your existing code.

Adding structure with spans

Without spans you get a flat list of calls. Spans let you group calls into named steps:

import llm_agent_trace

llm_agent_trace.patch()

@llm_agent_trace.span("summarize")
def summarize(text):
    return llm.invoke(f"Summarize: {text}")

@llm_agent_trace.span("verify")
def verify(summary):
    return llm.invoke(f"Is this accurate? {summary}")

with llm_agent_trace.session("research_agent"):
    summary = summarize(long_document)
    verdict = verify(summary)

Spans also work as context managers:

with llm_agent_trace.session("agent"):
    with llm_agent_trace.span("step_1"):
        result = llm.invoke(...)
    with llm_agent_trace.span("step_2"):
        result = llm.invoke(...)

Save trace as JSON

with llm_agent_trace.session("my_agent", output="trace.json"):
    ...

{
  "session_id": "a3f2b1",
  "name": "my_agent",
  "duration_ms": 3400,
  "total_calls": 2,
  "total_tokens_in": 852,
  "total_tokens_out": 297,
  "total_cost_usd": 0.0022,
  "spans": [...],
  "orphan_calls": [...]
}

Disable terminal output

with llm_agent_trace.session("my_agent", print_trace=False, output="trace.json"):
    ...

Custom cost table

llm_agent_trace.patch(cost_table={
    "my-fine-tuned-model": (0.005, 0.015),  # per 1k tokens: in, out
})

Introspection

import llm_agent_trace

sess = llm_agent_trace.current_session()   # active Session or None
sp   = llm_agent_trace.current_span()      # active Span or None

Supported providers (auto-detected)

Any provider not listed but using HTTP/REST is captured as unknown.

Limitations (v1)

• Streaming responses are logged as a single event when the stream closes; per-chunk tracing is not yet supported.
• gRPC-based providers are not supported (rare in practice — all major providers use HTTP).
• Monkey-patching works best when patch() is called before your LLM library imports. If you import from litellm import completion before calling patch(), that specific reference won't be intercepted — use litellm.completion(...) instead.

Contributing

Issues and PRs welcome. Please open an issue before starting work on a large change.

License

MIT