LLM Usage Framework

A generic, provider-independent Python framework for tracking LLM usage.

The framework provides a common interface for executing LLM requests and collecting standardized usage telemetry across different LLM providers.

The framework is storage-agnostic. It does not create database tables, manage databases, or require a specific persistence technology.

Applications consuming the package decide how usage data should be stored or exported.

Features

The framework captures:

• Input tokens
• Output tokens
• Total tokens
• Input character count
• Output character count
• Token source
• Provider
• Model
• Request ID
• Session ID
• Latency
• Project
• Service
• Feature
• Custom metadata

Additional capabilities include:

• Generic provider interface
• Provider registry
• Lazy provider loading
• Optional tokenizer support
• Token counting fallback
• Character-based token estimation
• Usage callback
• Provider-specific adapters
• Provider exceptions are propagated to the consuming application
• No persistence dependency

Supported Providers

Providers are implemented as adapters behind the same generic LLMTracker interface.

Architecture

Application
     |
     v
LLMTracker
     |
     v
Provider Registry
     |
     v
Provider Adapter
     |
     +---- Ollama
     +---- OpenAI
     +---- Anthropic
     +---- Gemini
     +---- Bedrock
     |
     v
Standardized LLMResponse
     |
     +---- Application Database
     +---- Kafka
     +---- Prometheus
     +---- Logging
     +---- Other Systems