llmtracer-sdk 2.0.0

Auto-track LLM cost, latency, and usage. Two lines of code, every provider.

LLM Tracer — Python SDK

Track cost, latency, and usage across all your LLM providers in one dashboard.

Works with OpenAI, Anthropic, Google, and any provider accessible through LangChain.

Install

pip install llmtracer-sdk

Quick Start — Global Handler (recommended)

Three lines. Every LLM call captured automatically — no changes to your chains or agents.

from llmtracer import LLMTracerCallbackHandler
from langchain_core.globals import set_handler

tracer = LLMTracerCallbackHandler(api_key="lt_...")
set_handler(tracer)

That's it. Every LLM call in your application — ChatOpenAI, ChatAnthropic, ChatGoogleGenerativeAI — is automatically tracked with model, tokens, cost, and latency. No need to pass callbacks=[tracer] to individual chains or agents.

View your dashboard at llmtracer.dev.

Environment variable pattern

import os
from llmtracer import LLMTracerCallbackHandler
from langchain_core.globals import set_handler

tracer = LLMTracerCallbackHandler(
    api_key=os.environ["LLMTRACER_API_KEY"],
    tags={"env": os.environ.get("ENVIRONMENT", "development")},
    enabled=os.environ.get("LLMTRACER_ENABLED", "true").lower() == "true",
)
set_handler(tracer)

Advanced — Per-Invocation Callbacks

Use per-invocation callbacks when you need to attach per-call tags (e.g., feature, user, customer) to specific chains or agents:

from llmtracer import LLMTracerCallbackHandler

tracer = LLMTracerCallbackHandler(
    api_key="lt_...",
    tags={"env": "production"}
)

result = await agent.ainvoke(
    input_data,
    config={
        "callbacks": [tracer],
        "metadata": {
            "llmtracer_tags": {
                "feature": "chat",
                "user_id": "u_sarah",
                "customer": "acme-corp",
            }
        }
    }
)

Tagging Patterns

Pattern	Tag	Example
Track cost by feature	feature	"chat", "search", "summarize"
Track cost by user	user_id	"u_sarah", "u_mike"
Track cost by customer (B2B)	customer	"acme-corp", "initech"
Track cost by conversation	conversation_id	"conv_abc123"
Track environment	env	"production", "staging"

Quick Start — Direct (no LangChain)

If you're calling provider SDKs directly:

from llmtracer import LLMTracer
from openai import OpenAI

tracer = LLMTracer(api_key="lt_...", tags={"env": "production"})

client = OpenAI()
tracer.instrument_openai(client)

# All calls are now tracked automatically
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    llmtracer={"tags": {"feature": "chat"}}  # optional per-call tags
)

Multi-provider

from openai import OpenAI
from anthropic import Anthropic

tracer = LLMTracer(api_key="lt_...", tags={"env": "production"})

openai_client = OpenAI()
anthropic_client = Anthropic()

tracer.instrument_openai(openai_client)
tracer.instrument_anthropic(anthropic_client)

# Both are now tracked in the same dashboard

Supported Providers

Provider	LangChain Callback	Direct Instrumentor
OpenAI	✅	✅
Anthropic	✅	✅
Google GenAI	✅	✅
Azure OpenAI	✅	Coming soon
AWS Bedrock	✅	Coming soon
Any LangChain LLM	✅	—

Configuration

tracer = LLMTracerCallbackHandler(
    api_key="lt_...",           # Required — from llmtracer.dev/settings
    tags={"env": "production"}, # Optional — global tags
    flush_interval=5.0,         # Seconds between batch flushes (default: 5)
    batch_size=50,              # Events per batch (default: 50)
    enabled=True,               # Set False to disable without removing code
)

How It Works

1. The callback handler hooks into LangChain's on_chat_model_start / on_llm_end lifecycle
2. It captures: model name, provider, input/output tokens, latency, success/failure
3. Events are buffered in memory and flushed every 5 seconds (or at batch_size)
4. Flush happens in a background thread — zero latency impact on your LLM calls
5. Events are POSTed to the LLM Tracer Cloud Function which calculates cost server-side

Zero Dependencies

The core SDK uses only Python stdlib (urllib.request, threading, hashlib).

LangChain callback mode requires langchain-core (which you already have if you're using LangChain).

License

MIT