llm-sketchkit

llm-sketchkit is a Go and Python library for measuring LLM usage. It counts distinct users and ranks the heaviest users, sessions or keys by tokens, and keeps the results in small summary files you can store, combine and share.

Use it to answer how many distinct users your LLM app has, and who uses the most tokens, across services, days and tools:

• Fast: 0.026 ms or less per range query, measured as a batch average.
• Cheap to keep: 305 days in 2.1 MB vs 1.25 GB of source records.
• Right when combined: Two tools with 46 and 26 users add up to 72; combined correctly, it's 52.
• Portable: Go and Python, same bytes, shared conformance vectors.
• Private by design: Keyed hashing before counting, no raw IDs in the files; keyed and pseudonymous, not anonymous.

Go-to-Python notebook and walkthrough.

Use It To

• Tracking tokenmaxxing? Whether you run a usage leaderboard or set per-team AI budgets, rank the heaviest users by token volume, with guaranteed bounds, from keyed IDs rather than a raw per-employee table. Then see how concentrated the usage is. Runnable example.
• Count distinct users across days, workers and services without counting the same user twice.
• Compare overlap between sets of prompts, tools or documents.
• Share and merge measurements across Go and Python while keeping raw identities at the source.

Install

Python 3.11-3.14; Go 1.26 or 1.27 with the latest security patch for main (published 0.2.x releases also support Go 1.25).

python -m pip install llm-sketchkit
# Go: inside a module (run "go mod init example" in a new directory first).
go get github.com/llm-measurement/llm-sketchkit/go/sketchkit/...@latest

Python

Generate a process secret, then count users and rank token consumption:

export LLM_SKETCHKIT_SECRET="$(python -c 'import secrets; print(secrets.token_hex(32))')"
python - <<'PY'
from llm_sketchkit import USER_V1, canonicalize_text_v1, frequentitems, hash64, hllpp, secret_from_env

secret = secret_from_env("LLM_SKETCHKIT_SECRET")
users = hllpp.new("small", USER_V1)
heaviest = frequentitems.Sketch("small", USER_V1)
for user_id, tokens in [("ana", 1_200), ("ben", 300), ("ana", 2_400), ("cy", 900)]:
    digest = hash64(secret, USER_V1, canonicalize_text_v1(user_id))
    users.add_hash(digest)
    heaviest.add_hash(digest, tokens)
top = max(heaviest.frequent_items(frequentitems.NO_FALSE_NEGATIVES), key=lambda i: i.lower_bound)
print(f"distinct users: {users.estimate():.0f}; heaviest user: {top.lower_bound:,} tokens")
PY

Output: distinct users: 3; heaviest user: 3,600 tokens.

Go

With the same secret, put this in main.go and run go run .. These user IDs are already canonical ASCII.

package main

import (
	"fmt"
	"log"

	sketchhash "github.com/llm-measurement/llm-sketchkit/go/sketchkit/hash"
	"github.com/llm-measurement/llm-sketchkit/go/sketchkit/hllpp"
)

func main() {
	secret, err := sketchhash.SecretFromEnv("LLM_SKETCHKIT_SECRET")
	if err != nil {
		log.Fatal(err)
	}
	users, err := hllpp.New(hllpp.ProfileSmall, sketchhash.UserV1, sketchhash.HMACSHA25664)
	if err != nil {
		log.Fatal(err)
	}
	for _, userID := range []string{"ana", "ben", "ana", "cy"} {
		digest, err := sketchhash.Hash64(secret, sketchhash.UserV1, []byte(userID))
		if err != nil {
			log.Fatal(err)
		}
		users.AddHash(digest)
	}
	fmt.Printf("distinct users: %.0f\n", users.Estimate())
}

Output: distinct users: 3. See Go canonicalization and the token-ranking gateway example.

Choosing An Integration

Limits

Distinct counts and similarity estimates have statistical error; ranking bounds are deterministic. The opening measurements describe one machine and one lab's released Claude Code and Codex records. Query times are batch averages over daily states already in memory; the whole-period distinct estimate rounds to 52. The summary measurements and whole-period merge check record the inputs, methods and commands. Library microbenchmarks have their own machine and workload details in the report.

Reported tokens describe usage rather than billing. For concurrent access, follow the ownership rules.

Documentation

• How it works and evidence at a glance.
• Usage guide: questions, token-heavy keys and merging.
• Go and the gateway example, Go-to-Python notebook and summary exchange.
• Security and privacy, wire compatibility and API reference.
• Development and contributing, FAQ and all documentation.

Status: Pre-1.0. See the changelog, supported versions and compatibility policy.

Feedback: Open an issue, contribute, or report a vulnerability privately.