optitokenize 0.1.0

Local-only CLI that prices Claude Code token usage and attributes cost to your repo's modules.

optitokenize

Local-only economics for your Claude Code sessions. optitokenize reads the transcripts Claude Code already writes to disk, prices the recorded token usage, and attributes that cost to the modules of your current git repository — so you can see where your agent spend went and what survived into commits.

• Local-only. Zero network calls at runtime. No telemetry, no update checks. Everything runs against files already on your machine.
• Read-only. Your transcripts and your git repo are never modified.
• Honest. Costs are recomputed from recorded token usage against vendored list prices — never fabricated, never silently substituted.

optitokenize is a measurement tool, not a scoreboard. Attribution is a heuristic. A module that costs a lot per accepted change is usually a hard module, not a badly written one — read the numbers as a map of where the work was, not a verdict on code quality.

New here? PRODUCT.md is a non-technical overview of what optitokenize does and who it's for. This README covers installation, commands, and exact behavior.

---

Install

Requires Python ≥ 3.11 and uv.

$ uv sync
$ uv run optitokenize doctor

Or build and install the wheel:

$ uv build
$ uv tool install dist/optitokenize-0.1.0-py3-none-any.whl

optitokenize has exactly two runtime dependencies: click and rich.

Quickstart

Run it from inside any git repository you've worked on with Claude Code:

$ optitokenize report --since 30d

optitokenize maps the current repo to its Claude Code transcript directory under ~/.claude/projects/, parses the sessions, and prints a per-module breakdown.

What it measures

For each module (by default the first two path components under the repo root), optitokenize reports:

Column	Meaning
SPEND	Total attributed cost of turns that touched files in the module.
ACCEPTED	Number of agent edit events in the module whose content survived into a commit.
$/DIFF	SPEND ÷ ACCEPTED — dollars per accepted change. Shown as — when nothing was accepted.
COMP. TAX	Comprehension tax: how much the agent had to read relative to what it wrote (see below).
REWORK	Fraction of edits in the module that were re-edited in the same file, same session.

Below the module table, three buckets account for spend that isn't code: (conversation) (turns with no file tool calls), (bash) (shell turns), and (outside repo) (paths outside the repo root). Module spend plus bucket spend always sums to the reported total.

A note on "Comp. tax"

Conceptually, comprehension tax is the ratio of input tokens the agent consumed to the output tokens it produced — a module you have to re-read a lot to change is expensive to comprehend. But billed input includes the entire conversation context, so a module touched late in a long session looks "unreadable" regardless of its code. optitokenize therefore defaults to a module-local, bytes-based ratio — bytes the agent read from the module's files divided by bytes it wrote to them — and offers the literal billed ratio via --comp-tax=billed. See docs/METRICS.md for exact definitions and caveats.

A note on cost

The number is a list-price reconstruction of the token usage recorded in your transcripts (cache reads priced at 10% of input, cache writes at their 1.25× / 2× premiums). For API-key users it closely matches the billed amount; for Pro/Max subscription users it is not an invoice — it's what the same usage would have cost at API list prices. Prices are vendored in src/optitokenize/pricing_data.json with their retrieval date; override with --pricing FILE.

Example

optitokenize report — /home/you/acme-api
window: 2026-06-24 → 2026-07-23 · 34 sessions · 1204 turns
total spend: $128.40 · cache hit rate: 71%

MODULE            SPEND  ACCEPTED  $/DIFF  COMP. TAX  REWORK
src/api          $52.10        21   $2.48       3.1x     19%
src/db           $22.60         4   $5.65       7.4x     50%
src/parser       $18.90        11   $1.72       2.2x      9%
tests/api        $11.30         8   $1.41       1.4x      0%
(root)            $3.20         2   $1.60       1.1x      0%

(conversation)   $12.10
(bash)            $6.80
(outside repo)    $1.40

warnings:
  src/db costs 3.3x the repo median per accepted change
  src/db comprehension tax is 3.4x the repo median

Attribution is heuristic — compare --attribution=uniform. Cost reflects task difficulty as well as code quality.

When a module has zero accepted edits, $/DIFF shows — — and the module is still listed. Those are often the interesting ones: spend that never landed.

Commands

optitokenize report  [--since 30d|YYYY-MM-DD] [--logs PATH] [--attribution bytes|uniform]
                    [--depth N] [--comp-tax readwrite|billed] [--match-threshold F]
                    [--pricing FILE] [--no-git] [--lenient] [--json] [--plain] [--no-prompts]
optitokenize cache | trend | explain <path> | doctor        # inherit the relevant flags

• report — the per-module economics table above.
• cache — modules ranked by full-rate (uncached) input spend, plus sessions with a cache-hit rate below 0.3 flagged as offenders.
• trend — ISO-week buckets of total spend, accepted diffs, and $/diff.
• explain <path> — one line per session that touched the module: date, short session id, spend, files touched, accepted/rejected/pending counts, and a truncated preview of the first prompt (disable with --no-prompts).
• doctor — what optitokenize found: transcript directories, session/turn counts, Claude Code versions seen versus those it's tested against, parse failures, models missing from the pricing table, git availability, and repo detection.

Useful flags

• --since 30d or --since 2026-06-01 — bound the reporting window.
• --attribution bytes (default) splits a turn's cost by bytes moved per file; --attribution uniform splits evenly. Compare the two — if a module's rank swings, its attribution is soft.
• --depth N — module granularity (path components under the repo root).
• --no-git — skip acceptance inference; ACCEPTED and $/DIFF render as —.
• --lenient — skip unparseable transcript lines and report a skip count instead of failing. Strict is the default.
• --json — stable, versioned machine-readable output ("schema": 1).
• --plain — deterministic plain-text tables (no colour, no box drawing).
• --no-prompts — never print any prompt text.

Exit codes

Code	Meaning
0	Success.
1	Usage error (bad flag value).
2	No transcripts found for this repo.
3	Parse or pricing failure (strict).

Privacy

Transcripts contain your prompts and your source code. optitokenize never writes transcript content anywhere. The only place prompt text is displayed is explain's first-message preview, truncated to 80 characters and suppressed entirely by --no-prompts. There is no cache, no log file, no upload.

How it works

1. Discover — map the current git root to its transcript directory under ~/.claude/projects/ (override with --logs PATH).
2. Parse — stream each session's JSONL into typed turns and edit events, de-duplicating records that resumed sessions rewrite.
3. Price — recompute each turn's cost from its recorded token usage against vendored list prices. Any costUSD in the transcript is ignored.
4. Attribute — attach each turn's cost to the file paths its tool calls referenced, roll paths up into modules, and set aside conversation/bash/ outside-repo spend into buckets.
5. Match — infer which edit events survived into commits using content-based matching (normalized to survive formatter mutations), so squash-merges and rebases don't break attribution.
6. Report — compute the metrics and render.

Format drift is the main risk: Claude Code's transcript format is not a stable public contract. optitokenize fails loudly on record shapes it doesn't recognize (naming the file, line, and Claude Code version), tracks the versions it's tested against in SUPPORTED.md, and surfaces all of this through doctor. See docs/FORMAT.md for the observed format and docs/DECISIONS.md for dated deviations from the design.

Acknowledgements

optitokenize is built with two excellent libraries:

• click (BSD-3-Clause) — the command-line interface.
• rich (MIT) — the terminal tables and formatting.

Developed with ruff, mypy, pytest, and uv. Every dependency is permissively licensed; see NOTICE for the full list.

License

MIT. See LICENSE.

Trademarks and disclaimer

optitokenize is an independent, community project. It is not affiliated with, sponsored by, or endorsed by Anthropic. "Claude", "Claude Code", and "Anthropic" are trademarks of Anthropic, used here only descriptively to identify the tool whose local session transcripts optitokenize reads. Pricing figures are reconstructed from Anthropic's public list prices (see pricing_data.json for the source and retrieval date) and are informational, not an invoice.