tokenwatcher 0.1.1

Python SDK for monitoring LLM API calls with TokenWatcher

TokenWatcher Python SDK

Monitor your LLM API calls with zero code changes. Simply wrap your OpenAI or Anthropic client and get instant observability into tokens, costs, latency, and errors.

Installation

pip install tokenwatcher

For OpenAI support:

pip install tokenwatcher[openai]

For Anthropic support:

pip install tokenwatcher[anthropic]

For both:

pip install tokenwatcher[all]

Quick Start

OpenAI

from tokenwatcher import MonitoredOpenAI
from openai import OpenAI

# Wrap your client (that's it!)
client = MonitoredOpenAI(
    OpenAI(api_key="sk-..."),
    api_key="ym_your_monitoring_key"
)

# Use OpenAI exactly as before
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello!"}]
)

Anthropic

from tokenwatcher import MonitoredAnthropic
from anthropic import Anthropic

# Wrap your client
client = MonitoredAnthropic(
    Anthropic(api_key="sk-ant-..."),
    api_key="ym_your_monitoring_key"
)

# Use Anthropic exactly as before
response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}]
)

Configuration

from tokenwatcher import MonitoredOpenAI

client = MonitoredOpenAI(
    openai_client,
    api_key="ym_your_key",           # Required: Your TokenWatcher API key
    base_url="https://api.token-watcher.com",  # Optional: Custom backend URL
    context="production/chatbot",     # Optional: Tag events with context
    user_identifier="user-123",       # Optional: Associate events with a user
    buffer_size=100,                  # Optional: Max events before auto-flush (default: 100)
    flush_interval=10,                # Optional: Seconds between auto-flush (default: 10)
    enabled=True,                     # Optional: Disable monitoring (default: True)
)

Features

• Zero Code Changes: Drop-in replacement for OpenAI and Anthropic clients
• Automatic Monitoring: Captures tokens, latency, costs, and errors
• Auto-Context Detection: Automatically detects calling module and function for better observability
• Buffered & Async: Events are buffered and sent in background, no performance impact
• Silent Failures: If monitoring fails, your LLM calls continue uninterrupted
• Multi-tenant: Each API key is isolated to your account
• Type Safe: Full type hints for Python 3.8+

How It Works

The SDK wraps your LLM client and intercepts API calls to extract:

• Provider and model used
• Input and output token counts
• Request latency
• Success/error status
• Error details (if any)
• Calling context (automatically detects module and function name)

Events are buffered and sent to your TokenWatcher backend in batches, ensuring minimal overhead.

Automatic Context Detection

When you don't provide an explicit context parameter, the SDK automatically detects the calling module and function:

# In myapp/chatbot.py
def handle_user_message(message):
    client = MonitoredOpenAI(openai_client, api_key="ym_key")
    response = client.chat.completions.create(...)
    # Context will be auto-set to: "myapp.chatbot.handle_user_message"

This provides automatic code-level observability without any manual tagging! You can still override this by providing your own context parameter.

Requirements

• Python 3.8+
• requests >= 2.25.0
• openai >= 1.0.0 (optional, for OpenAI support)
• anthropic >= 0.18.0 (optional, for Anthropic support)

Environment Variables

You can configure the SDK using environment variables:

export TOKENWATCHER_API_KEY=ym_your_key
export TOKENWATCHER_BASE_URL=https://api.token-watcher.com
export TOKENWATCHER_CONTEXT=production

Then use without explicit configuration:

from tokenwatcher import MonitoredOpenAI
from openai import OpenAI
import os

client = MonitoredOpenAI(
    OpenAI(api_key="sk-..."),
    api_key=os.getenv("TOKENWATCHER_API_KEY")
)

Support

• Documentation: https://docs.token-watcher.com
• Issues: https://github.com/tokenwatcher/tokenwatcher-python/issues
• Email: [e-mail]

License

MIT License - see LICENSE file for details.