tknops-llm 0.0.1

Precise Token Tracking SDK for tknOps

tknOps Python SDK

Precise Token Tracking for Multi-Tenant SaaS.

The tknops-llm SDK provides full visibility into AI costs per user, team, and organization. It automatically captures tokens, cost, and latency from your LLM calls.

Installation

pip install .
# OR
pip install tknops-llm

Initialization

Initialize the client with your API Key (from the dashboard) and the Analzyer URL.

from tknops_llm.client import AIAnalytics

# Initialize the client
# The client runs a background thread to batch/send events asynchronously.
tracker = AIAnalytics(
    api_key="your_api_key_here",
    base_url="http://localhost:8000" # Update with your deployed analyzer URL
)

Usage

1. Manual Tracking

If you manually calculate tokens or want to log generic events:

tracker.track(
    user_id="user-123",
    model="gpt-4",
    provider="openai",
    input_tokens=50,
    output_tokens=120,
    cost_usd=0.004,
    latency_ms=1500,
    tags=["production", "chatbot"],
    metadata={"conversation_id": "abc-123"},
    prompt_text="Hello, how are you?",
    response_text="I'm doing well, thank you!"
)

2. Automatic OpenAI/LangChain Response Tracking

The SDK can automatically extract metrics from standard response objects.

OpenAI Example:

import openai

response = openai.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Say hello!"}]
)

tracker.track_response(
    response=response,
    user_id="user-123",
    cost_per_1k_input=0.0015,
    cost_per_1k_output=0.002,
    tags=["test"]
)

LangChain Example:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4")
response = llm.invoke("Tell me a joke")

tracker.track_response(
    response=response,
    user_id="user-123",
    tags=["langchain"]
)

Shutdown

The client uses a daemon thread. To ensure all pending events are flushed before your script exits, call:

tracker.shutdown()