tokenfair-digest 0.1.0

Client-side digest tool: tokenizes LLM logs locally on the shared ruler and emits metrics-only JSON. No text ever leaves the machine.

tokenfair-digest

Client-side digest tool for LLM API billing audits. Captures SDK responses, recounts tokens locally on a shared cl100k_base ruler, and emits metrics-only JSON. No prompt or completion text ever leaves your machine — only token counts are written.

Install

pip install tokenfair-digest

Capture (drop-in, one line)

Add a single line where you create your client, and every response is logged to a local JSONL file.

OpenAI

from openai import OpenAI
from tokenfair_digest.capture import capture_openai

client = capture_openai(OpenAI(), "openai_logs.jsonl")
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
)
# openai_logs.jsonl now has one record per response

Anthropic

from anthropic import Anthropic
from tokenfair_digest.capture import capture_anthropic

client = capture_anthropic(Anthropic(), "anthropic_logs.jsonl")
response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)

Generate a digest

Once you have a JSONL log file, convert it to a metrics-only digest:

tokenfair-digest openai_logs.jsonl -f openai -o digest.json

Supported formats: openai, anthropic, gemini, atap

The digest JSON can be opened in the TokenFair web app or VS Code extension to audit where your tokens went.

Programmatic usage

from tokenfair_digest import build_digest, count_tokens, CaptureWriter

# Count tokens on the shared ruler
n = count_tokens("Hello, world!")

# Build a digest from parsed records
from tokenfair_digest.parsers import parse_openai_export
with open("openai_logs.jsonl") as fh:
    digest = build_digest(parse_openai_export(fh))

Guarantees

• No text on disk. Only metric fields are written (record_id, provider, model, token counts). Prompt and completion text stay in memory only.
• Never breaks your call. The capture tee swallows any logging error — your API call is unaffected.
• Thread-safe. Writes are serialized with a lock.
• Streaming is skipped, not corrupted. Streamed responses have no usage to record.
• Matches the JS side. Uses the same cl100k_base tokenizer as @garzillion-labs/capture, so counts line up.

License

MIT