tokli

Report-only usage tracking for tokli — a cost & usage dashboard for AI APIs. Wrap your existing provider client, keep calling it exactly as before, and tokli reports the token usage in the background. We never see your provider API key and we never sit in your request's critical path.

Supports OpenAI, Anthropic, Gemini, DeepSeek, xAI, Mistral, Qwen, GLM, Kimi and OpenRouter.

Install

pip install tokli

openai, anthropic and google-genai are optional extras. tokli has zero runtime dependencies and never pulls a provider SDK, or a version bound, into your app.

Quickstart

from openai import OpenAI
from tokli import wrap_openai

client = wrap_openai(OpenAI())  # ingest key from TOKLI_INGEST_KEY
client.chat.completions.create(model="gpt-5.4", messages=messages)

The returned client behaves exactly like the original: every attribute we don't instrument passes straight through, isinstance(client, OpenAI) still holds, and client.with_options(...) / client.copy(...) hand back clients that are still instrumented.

Without an ingest key (neither ingest_key nor TOKLI_INGEST_KEY), the wrapper is a silent no-op: your app keeps working exactly as before, nothing is reported.

Async

The same function. There is no separate async wrapper — wrap_openai(OpenAI()) and wrap_openai(AsyncOpenAI()) return the same kind of object.

from openai import AsyncOpenAI
from tokli import wrap_openai

client = wrap_openai(AsyncOpenAI())
await client.chat.completions.create(model="gpt-5.4", messages=messages)

Per-feature attribution

checkout = client.with_feature("checkout")  # immutable and chainable
await checkout.chat.completions.create(...)  # every event tagged "checkout"

with_feature exists at runtime but type checkers cannot see it, because Python has no intersection types and the wrappers return your client's own type so autocompletion survives. For a type-checked equivalent, use the free function:

from tokli import with_feature

checkout = with_feature(client, "checkout")

Available wrappers

wrap_openai, wrap_anthropic, wrap_gemini, wrap_deepseek, wrap_xai, wrap_mistral, wrap_qwen, wrap_glm, wrap_kimi, wrap_openrouter — all share the same (client, **options) -> client signature and the .with_feature(tag) chaining above. The seven OpenAI-compatible ones take the openai package pointed at the provider's base URL; each wrapper's docstring carries the exact URL and the provider's caching quirks.

Config

An argument beats the environment variable, which beats the default. An unusable value (a typo in an env var) is skipped rather than raised — a misconfiguration must not take your app down.

on_error runs on a worker thread. Keep it thread-safe and don't touch request-scoped state from it. It receives (error, reason), where reason is "transport", "no_usage" or "parse".

flush_ms is per-wrap but the worker pool is per-process. If two clients are wrapped with different values, the pool waits for the longest one — a short deadline must not discard another wrapper's pending events. So flush_ms=0 only disables the wait if it is the only value configured in the process.

What gets reported

We instrument the calls that bill tokens, and only those:

Streaming and non-streaming are both covered, as is with_options() / copy(), which re-wrap the client they return.

What is not reported

These pass through untouched. If you use one, its spend will not appear in tokli — this list is the whole of it. For the namespaces we do instrument, a test enumerates every public method and fails the moment a provider SDK grows one nobody has classified.

Background responses. responses.create(background=True) returns immediately with no usage, so there is nothing to report at that point — the real numbers arrive later through responses.retrieve(), which we deliberately leave uninstrumented (it is idempotent, and reporting it would count the same response on every poll). A background call that also streams is reported: its terminal event carries the usage. If you rely on non-streaming background calls, report those yourself.

Streaming

For streaming chat completions on OpenAI, DeepSeek and Qwen, pass stream_options={"include_usage": True} so the final chunk includes token usage — without it there is nothing for tokli to report. xAI, GLM, Kimi and OpenRouter send it either way. Mistral takes no stream_options at all — its schema rejects unknown fields. Anthropic, Gemini and the Responses API need no flag.

If you abandon a stream part-way (break out of the loop), nothing is reported: we cannot know what you were billed, and we would rather report nothing than guess.

How it works

Your code keeps calling the provider with your own key. After the response comes back, the SDK reads the usage object already present in it and hands those raw numbers to a small background worker pool — it never blocks your request, your event loop, or your key. Cost is computed server-side against a versioned price table, never by the SDK.

Reporting can never break your app: if anything in our own path raises — including your on_error handler — you still get the provider's response untouched.

A few consequences worth knowing:

• The report POST reuses one keep-alive connection per worker thread. A connection idle for more than 5 seconds is replaced rather than reused, and a failed send is dropped, never retried: a retry could duplicate an event if the server processed it before the socket broke, and a duplicate corrupts your cost far worse than a lost event does.
• http_proxy / https_proxy are not read. tokli talks to its own ingest endpoint over http.client, which ignores proxy environment variables — no surprise routing. Corporate proxy support is not in v1.
• TLS verification is always on and cannot be disabled.
• At exit the SDK waits up to flush_ms for pending reports, so short scripts and CLI jobs don't systematically lose their last event.
• After fork() (gunicorn/uvicorn --preload) the child drops the queue it inherited, so workers never resend the parent's backlog.

Requirements

Python 3.10+.

Links

• tokli.dev
• Source & issues