LLM Monitor SDK

Lightweight SDK for monitoring LLM cost, latency, and token usage.

The LLM Monitor SDK automatically intercepts LLM API calls (OpenAI, Anthropic, etc.), collects usage telemetry, and sends it to the monitoring backend so you can analyze:

• Token usage
• Cost per feature
• Latency metrics
• Model usage distribution
• Optimization suggestions

Once telemetry is sent, metrics can be viewed on the dashboard:

https://ai-operations-optimizer.vercel.app

---

Installation

Install the SDK using pip:

pip install llm-monitor-sdk

---

Quick Start

Import the SDK and wrap your OpenAI client.

import openai
from llm_monitor import LLMMonitor, feature_tag

monitor = LLMMonitor(
    api_key="YOUR_PROJECT_API_KEY",
    ingest_url="https://your-backend-domain/api/v1/ingest"
)

client = monitor.wrap_openai(openai.OpenAI())

Now every LLM request will automatically be monitored.

---

Basic Example

import openai
from llm_monitor import LLMMonitor, feature_tag

monitor = LLMMonitor(
    api_key="YOUR_PROJECT_API_KEY",
    ingest_url="https://your-backend-domain/api/v1/ingest"
)

client = monitor.wrap_openai(openai.OpenAI())

with feature_tag("summarization"):
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "user", "content": "Summarize the importance of AI monitoring."}
        ]
    )

print(response.choices[0].message.content)

The SDK will automatically capture:

• model used
• input tokens
• output tokens
• latency
• estimated cost
• feature tag

Telemetry is sent asynchronously to the monitoring backend.

---

View Metrics Dashboard

After running your application, view metrics at:

https://ai-operations-optimizer.vercel.app

The dashboard provides:

• Real-time LLM usage metrics
• Cost analysis
• Latency monitoring
• Feature-level hotspots
• Automated optimization suggestions

---

Feature Tagging

Feature tags allow you to track which part of your application generated LLM calls.

Example:

from llm_monitor import feature_tag

with feature_tag("chatbot"):
    client.chat.completions.create(...)

with feature_tag("classification"):
    client.chat.completions.create(...)

In the dashboard, metrics will appear grouped by feature.

---

How It Works

1. Your application calls an LLM API.
2. The SDK intercepts the request.
3. Usage data (tokens, latency, cost) is collected.
4. Events are batched and sent to the ingest API.
5. The backend stores telemetry and analyzes usage.
6. Metrics appear on the dashboard.

All telemetry shipping happens in a background thread to avoid blocking your application.

---

Configuration

Initialize the SDK with the following parameters:

LLMMonitor(
    api_key="PROJECT_API_KEY",
    ingest_url="https://your-backend-domain/api/v1/ingest"
)

Parameter	Description
api_key	Project API key used to authenticate telemetry ingestion
ingest_url	Backend endpoint receiving LLM usage events

---

Privacy

The SDK does not store prompt or response content by default. Only metadata is collected:

• token counts
• latency
• model name
• feature tag
• estimated cost

This ensures safe monitoring without exposing sensitive data.

---

Requirements

• Python 3.9+
• OpenAI Python SDK or compatible LLM client

Dependencies installed automatically:

• httpx
• wrapt
• tiktoken

---

Troubleshooting

If metrics do not appear:

1. Verify your api_key is correct.
2. Confirm ingest_url points to the running backend.
3. Ensure your application executed at least one LLM request.

---

License

MIT License

---

Dashboard

View LLM usage analytics and optimization suggestions:

https://ai-operations-optimizer.vercel.app

---