infertrack

Zero-config LLM call interceptor. Track token usage, cost, and latency — locally, no cloud required.

pip install infertrack

from infertrack import watchdog

@watchdog(tag="summarise")
def ask(prompt):
    return client.chat.completions.create(
        model="qwen2.5:0.5b",
        messages=[{"role": "user", "content": prompt}]
    )

ask("Summarise this document...")

$ watchdog summary
  infertrack summary  ·  Last 24h
────────────────────────────────────────────────────────────────────────
  Calls                  47
  Input tokens         82.3k
  Output tokens        46.1k
  Total tokens        128.4k
  Total cost          $0.4100
  Avg latency           834ms
  Success rate         97.9%
────────────────────────────────────────────────────────────────────────
  Models:
    gpt-4o                                   31 calls
    qwen2.5:0.5b                             12 calls
    gpt-4o-mini                               4 calls

Why infertrack?

You're building an LLM feature. You want to know: how many tokens did each call use, how long did it take, what did it cost, did it fail silently? Right now you add print statements. Or you install LangSmith and spend a day configuring it.

infertrack gives you a decorator and a CLI. No cloud account. No config files. No framework lock-in.

Installation

pip install infertrack          # core + CLI

Requires: Python 3.10+, click>=8.0, openai>=1.0 (optional, for OpenAI/Ollama)

Works with Ollama out of the box — free, local, no API key:

ollama pull qwen2.5:0.5b

from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

Usage

@watchdog decorator

Wrap any function that returns a raw API response object:

from infertrack import watchdog
from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

@watchdog(tag="summarise", user_id="alice")
def ask(prompt: str):
    return client.chat.completions.create(
        model="qwen2.5:0.5b",
        messages=[{"role": "user", "content": prompt}]
    )

response = ask("What is the capital of France?")
print(response.choices[0].message.content)
# Every call is now logged to ~/.infertrack/logs.db

Parameters:

watch() context manager

Track one or more API calls in a block and read live metrics afterward:

from infertrack import watch

with watch(tag="batch-summarise", user_id="alice") as w:
    for doc in documents:
        resp = client.chat.completions.create(
            model="qwen2.5:0.5b",
            messages=[{"role": "user", "content": doc}]
        )
        w.add_response(resp)

print(f"Used {w.tokens_used:,} tokens")
print(f"Cost: ${w.cost_usd:.6f}")
print(f"Latency: {w.latency_ms:.0f}ms")
print(f"Calls: {w.call_count}")

Budget() enforcement

Stop a runaway loop before it empties your wallet:

from infertrack import Budget, BudgetExceeded

try:
    with Budget(max_usd=0.10, user_id="alice") as b:
        for chunk in large_document_chunks:
            resp = client.chat.completions.create(
                model="gpt-4o",
                messages=[{"role": "user", "content": chunk}]
            )
            b.add_response(resp)   # raises BudgetExceeded if over $0.10
except BudgetExceeded as e:
    print(f"Stopped: {e.user_id} spent ${e.spent:.4f} (limit ${e.limit:.4f})")

Period options — control what prior spend counts against the budget:

Budget(max_usd=0.10, user_id="alice", period="today")    # default: since midnight UTC
Budget(max_usd=1.00, user_id="alice", period="all")      # all historical spend
Budget(max_usd=0.05, user_id="alice", period="session")  # this block only

FastAPI example:

@app.post("/generate")
async def generate(prompt: str, user_id: str):
    with Budget(max_usd=0.05, user_id=user_id) as b:
        resp = client.chat.completions.create(...)
        b.add_response(resp)
    return {"result": resp.choices[0].message.content}

CLI

All commands read from ~/.infertrack/logs.db by default. Override with --db /path/to/logs.db.

watchdog summary

watchdog summary                   # last 24h
watchdog summary --last 7d         # last 7 days
watchdog summary --last all        # all time

watchdog tail

watchdog tail                      # last 20 calls
watchdog tail -n 50                # last 50 calls
watchdog tail --tag summarise      # filter by tag
watchdog tail --user alice         # filter by user
watchdog tail --model gpt-4o       # filter by model

watchdog top

watchdog top                              # top tags by cost
watchdog top --by tokens                  # rank by token usage
watchdog top --by calls                   # rank by call count
watchdog top --group user                 # group by user_id
watchdog top --group model                # group by model
watchdog top --limit 5 --last 7d          # top 5 in last 7 days

Storage

All data is stored locally in ~/.infertrack/logs.db (SQLite). No data ever leaves your machine.

Override the path with the --db flag on any CLI command, or pass db_path= to any decorator/context manager.

Each log entry contains:

Supported models & pricing

Pricing is embedded in pricing/prices.json and used for cost calculation. Ollama models are always free.

Unknown models default to $0.00 cost and are still tracked for tokens and latency.

What infertrack does NOT do

• No prompt storage — the content of your messages is never logged
• No cloud sync — all data stays in your local SQLite file
• No network calls — zero outbound connections, ever
• No token counting library — tokens come from response.usage only (what the API reports)
• No framework required — works standalone alongside LangChain, LlamaIndex, or raw API calls

Roadmap

• @watchdog decorator
• watch() context manager
• Budget() enforcement
• CLI: summary, tail, top
• infertrack.intercept() — zero-code-change global monkey-patch (Day 7)
• Anthropic provider (Day 8)
• watchdog export --format csv/json (Day 9)
• Streaming response support (Day 10)

Development

git clone https://github.com/yourname/infertrack
cd infertrack
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

# Run tests (no Ollama needed)
pytest tests/unit/ -v

# Run integration tests (requires Ollama running)
pytest tests/integration/ -v -m integration

# Lint
ruff check src/

License

Apache