tokencur

The CUR for your tokens — an open-source pipeline that turns AI usage into FOCUS-conformant cost datasets, validated in CI by the FinOps Foundation's own validator. Today it reads the local logs of three coding agents — Claude Code, Codex CLI and Kimi Code — prices them against 290+ models, and adds unit economics, savings recommendations and a local ledger that keeps the history after the agents delete their logs.

A FinOps tool that practices FinOps on itself: the first dataset is my own real AI spend.

Why

• AI cost management is the #1 skill gap in FinOps (State of FinOps 2026: 98% of organizations now manage AI spend).
• FOCUS already supports tokens — the spec has columns for token/credit-based billing, with OpenAI usage as an official example. The standard is ready; open implementations are not.
• The FinOps Foundation's focus_converters covers AWS, GCP, Azure and OCI — zero AI providers. tokencur aims to contribute an AI-provider converter upstream.

Related work

Design principles

1. Privacy by construction — ingestion keeps usage metadata only (tokens, models, timestamps). Conversation content is never extracted.
2. Measure existing spend, don't generate spend to measure — the first data source is local Claude Code session logs, which already exist on disk. Budget: ~$0.
3. List-cost showback — subscription usage isn't billed per token, so costs are computed as API-equivalent list cost. Pricing has two layers: a curated, dated Anthropic rate card (pricing.py) that always wins, and a vendored snapshot of the community-maintained LiteLLM price database as fallback (290+ live models across Anthropic, OpenAI, Gemini, DeepSeek, Kimi/Moonshot, GLM/Z.ai and Ollama, refreshed daily by the price-watch action; models LiteLLM retires keep their last known rate, so historical usage stays priced). Unknown models surface as unpriced usage rather than silently costing $0. Rates are point-in-time: each call is valued at the list rate in force on its day, from a rate history the snapshot keeps. Request options that change the price are applied as Claude Code logs them: fast mode (2x), US-only inference (1.1x) and the Batch API (0.5x).
4. Explainable over clever — every line that ships is one the maintainer fully understands and can defend.
5. Reproducible figures — the report, the export, the outcomes views and the observatory name what produced each number: the tokencur version, the curated card's date and the pricing snapshot's SHA-256. Each release publishes and attests that same snapshot file, so anyone can rerun a figure on the exact prices behind it.

Architecture

flowchart LR
    A["Claude Code · Codex CLI · Kimi Code<br/>local logs"] -->|"ingesters<br/>(metadata only)"| L[("Ledger<br/>SQLite")]
    L --> P["Pricing<br/>curated card + LiteLLM snapshot"]
    W["price-watch action<br/>(daily)"] -.->|refreshes| P
    P --> R["report · recommend"]
    P --> F["FOCUS 1.2 normalizer"]
    F --> E["export: FOCUS CSV<br/>(validated in CI)"]
    F --> O["observatory · dashboard"]

Each design decision, with its context and its cost, is recorded as an architecture decision record.

One module per layer — ingest/ (one adapter per source), ledger, pricing, focus — with the commands on top. Every command loads records through one function, sources.load_records().

Quickstart

Requires Python 3.11+. No runtime dependencies.

pip install tokencur
tokencur report                # cost summary in your terminal
tokencur export focus.csv      # FOCUS 1.2 conformant dataset
tokencur recommend             # avoided cost + what-if headroom
tokencur export sept.csv --since 2026-09-01 --until 2026-10-01   # one billing period
tokencur doctor                # read-only health check: log formats, ledger, pricing
tokencur import runpod FILE    # billed cost: a RunPod billing export
tokencur outcomes [REPO...]    # usage value per commit, repository by repository
tokencur outcomes --sessions   # usage value per change that landed and stayed
tokencur limits                # how close the 5-hour and weekly plan limits are

tokencur --help lists every command, and python -m tokencur works the same. Under a negotiated contract, --discounts discounts.json (for example {"discounts": {"Anthropic": 0.15}}) keeps the public price in FOCUS ListCost and puts the contracted price in ContractedCost, EffectiveCost and BilledCost; the report adds a contracted total. The export stays in USD, the currency every provider bills in; tokencur report --currency MXN --fx-rate 18.37 also shows totals at a rate you give (nothing is fetched). Periods are UTC days with --until excluded, the way a billing period is cut.

With no arguments it scans every known local source on your machine — Claude Code (~/.claude/projects), Codex CLI (~/.codex/sessions) and Kimi Code (~/.kimi-code/sessions) — and prints per-model, per-source and per-day API-equivalent cost, including provider-correct cache economics. Each scan is also kept in a local ledger, so history survives when the agents delete their old logs.

For the visual version — daily trend, cost by model, token-type mix and unit economics, each view exposing the DuckDB SQL behind it:

From a clone of this repository:

pip install -e ".[dashboard]"
streamlit run src/tokencur/dashboard.py

Real output over the maintainer's own machine on 2026-10-02 (10.7k model calls from ~400 MB of logs, 1.8 s), trimmed:

model                                 calls       input     output     cache_read  cache_write  cost USD
claude-opus-5-5                       2,545       5,230  4,453,008  1,039,768,719   21,810,960    471.43
gpt-5.4                               2,790  34,683,046  1,807,104    304,527,488            0    189.95
gpt-5.3-codex                         5,010  26,148,134  1,997,552    377,040,128            0    139.71
claude-fable-5-1                         14         418     64,013      1,732,662      185,892      7.36
gpt-5.2-codex                           139     347,399     71,796      4,995,072            0      2.49
...

by source:
  claude-code   $482.51
  codex         $332.95
  kimi-code     $2.44

API-EQUIVALENT TOTAL (showback): $817.90

Performance

On the maintainer's machine the full pipeline reads 10.7k model calls from ~400 MB of real logs in 1.8 s. Real transcripts carry the conversations, which tokencur decodes but never keeps. For scale, scripts/benchmark.py times every stage on synthetic metadata-only logs (Python 3.13, Intel i7-1355U, Linux):

Time grows linearly. The benchmark holds the scanned and the stored history at once, so its peak memory is about twice what one command needs. Reproduce with python scripts/benchmark.py --messages 1000000 --files 2000.

The FOCUS export is the bottleneck, at about 14 µs per row, and every command holds the full history in memory. That suits one developer's logs. A team-scale pipeline would stream from the ledger in batches and write Parquet or DuckDB instead of one CSV; that is not built yet.

Ledger

Coding agents treat their logs as disposable: Claude Code deletes session transcripts after cleanupPeriodDays (30 days by default). Computed from the logs alone, totals would shrink as history disappears. So every command — report, export, recommend, the observatory and the dashboard — first keeps what it scans in a local SQLite ledger, then reports the ledger's full history:

• Deduplicated per usage event — re-running adds nothing, and a resumed session's copied messages count once.
• Nothing is forgotten — events whose logs are gone keep their last known values; events still on disk are refreshed from the latest parse.
• Metadata only — token counts, models, timestamps, workspace and session ids, the directory each call ran in, in a file readable by its owner only.
• Location — ~/.local/share/tokencur/ledger.sqlite3 (honours $XDG_DATA_HOME), or wherever $TOKENCUR_LEDGER points.
• Corrections are audited, not erased — the schema is versioned and migrated in place. Before an upgrade the file is copied to ledger.sqlite3.schema-N.bak, and rows a later version finds were not usage move to a superseded table with when and why. Schema 2 retired the Codex re-sent reports earlier versions double counted (see the changelog); schema 3 records each call's price-changing request options, schema 4 billed charges, schema 5 the directory each call ran in, schema 6 CI runs, and schema 7 the plan limits that turned requests away.

The ledger keeps only what it has seen: usage deleted before the first run is gone. An explicit path (python -m tokencur report ROOT) is reported as-is and never stored.

Billed cost

Showback values usage at list price; a bill is what a provider actually took. RunPod is the first billed source: scripts/fetch_runpod_billing.py saves the billing history from RunPod's REST API (the one step that goes online, when you run it), and tokencur import runpod FILE keeps it in the ledger. Reports show it as a separate total, "BILLED (real money, from provider bills)", the FOCUS export gains Compute rows whose BilledCost is the billed amount, and the observatory shows it as actual money outside subscription leverage (ADR 0009).

Unit economics: usage value per commit and per successful change

Cost alone does not say whether usage paid off. tokencur outcomes sets it against the commits it went into, per git repository:

repository                              days  calls  usage value  commits  agent   lines  per commit
~/Proyectos/tokencur  2026-09-24..2026-10-02    424       $70.78       40     40  10,824       $1.77

• Attribution by where each call ran. Agents log the working directory of every call, and the call belongs to the git repository that holds it. Usage outside any repository, or in a directory since deleted, is reported as unattributed, never spread.
• Your commits in the days with usage. Non-merge commits by your user.email (or --all-authors), within --since/--until or else the repository's days with agent usage. Agent-signed commits and lines changed are shown as context.
• Read-only and local. git runs on your repositories; nothing is sent.

A commit is a coarse unit: it measures output, not quality. The useful comparison is a repository with itself over time. Quality, latency and reliability are the next layers (ADR 0010); the plan is in #15.

tokencur outcomes --sessions goes one step further, toward cost per successful task: one row per agent session, with what a person put in and what stayed.

session   started (UTC)     repository            calls  usage value  prompts  API time  retries  landed  reverted  per success
58cb4580  2026-10-01 20:20  ~/Proyectos/tokencur    119       $16.99        9       36m     5.1%       6         0        $2.83

• Success means it stayed. A change succeeds when it reaches the default branch and is never reverted; a rebased copy is the same change. A repository whose work lands on another branch names it in its own git config, git config tokencur.branch motor-pixel, and the view says so.
• A change goes to the session behind it: the last one that worked in its repository in the day before the commit.
• What a person put in: prompts count the messages a person typed, and API time and retries come from Claude Code's own counters.

On tokencur's own repository that comes to $1.89 of usage value per successful change (ADR 0012).

CI on each change's own code

A change can land with its tests failing. To see which successes passed CI, capture a GitHub Actions workflow's runs and import them:

python scripts/fetch_github_ci.py --workflow ci.yml   # in the repository; goes online
tokencur import github-ci ~/.local/share/tokencur/raw/github-ci/<capture>.json

outcomes --sessions then adds a column with each session's changes that passed and failed, and two lines under the headline:

usage value per change that also passed its own CI: $7.40 ($192.30 over 26 changes)
CI tested the code of 30 of 61 changes: 26 passed, 4 failed; 31 tested only with later commits

• Its own code, nothing else. A run counts for a change when it tested exactly the change's code: the same git tree, which a rebase onto an unmoved base keeps. A result is never borrowed from a later commit, which would hide failures that a later commit fixed.
• Always with its coverage. A push tests only its newest commit, so CI covers part of the changes; the figure says how many.
• The script keeps nine fields per run (ids, event, status, conclusion, commit, tree, times): no commit message, name, email or branch, and no job log. A token is optional for public repositories and never written. tokencur itself stays offline (ADR 0013).

When an agent changes its logs

Agents change their log formats without notice. tokencur doctor scans every source read-only and reports files, usage lines, records, unreadable lines and the agent versions the logs name. It flags what looks like a format change: log files with no usage lines, usage lines that yield no records, or more than 1% unreadable lines. It also checks the ledger's schema and SQLite integrity and prints the pricing snapshot's SHA-256. It exits 1 when anything needs attention. A malformed line is skipped, never guessed at, and never stops a scan.

It also reconciles tokencur with Claude Code's own accounting. When a session ends, Claude Code writes the cost it counted for each model, including calls it never writes to the transcript. doctor compares that with tokencur's value of the same sessions, and flags a gap either way: below 85% means calls are going missing, above 105% means calls are being counted twice.

Plan limits: how much of a subscription is left

A subscription such as Claude Max caps usage in a five-hour window and a weekly one, and states neither cap in tokens or dollars. Claude Code does log, as structured metadata, each time a limit turned a request away and when that limit resets. tokencur limits measures every window in API-equivalent usage value:

week: $111.07 = 10% of the lowest ceiling since Wed 2026-10-14 05:00 CST; resets Wed 2026-10-21 05:00 CST (in 6d 14h)
  limit hit on this plan at $1,063.07 (Tue 2026-10-13 21:46 CST)
  at the last 7 days' pace ($146.71/day) the week reaches the lowest ceiling seen ($1,063.07) around Wed 2026-10-21 02:22 CST, 2h 37m before it resets
  also hit at $1,351.63 (Tue 2026-10-06 18:43 CST), in a window that began on another plan: not a reference

5 hours: $111.07 = 82% of the lowest ceiling since Wed 2026-10-14 11:00 CST (window inferred from call times); resets Wed 2026-10-14 16:00 CST (in 1h 21m)
  limit hit on this plan at $135.05 (Wed 2026-10-07 13:17 CST), $148.30 (Sat 2026-10-03 18:15 CST)

• A ceiling is the usage value a window had used when its limit turned a request away, filed under the plan in force that day. Plans come from subscriptions.json in the current directory, else ~/.config/tokencur/subscriptions.json, or the file --plans names; their days are read in your time zone, in the formats the observatory reads. It is a reference for the current plan only when its whole window ran on that plan and these logs hold its usage.
• The week is placed by the latest weekly reset on record, since weekly limits reset on a fixed schedule. Five-hour windows open with the first call after the last one closed, rounded down to ten minutes (every reset logged so far matches that rule); past a logged reset they are rebuilt from call times.
• A gauge, not a balance. Limits are not priced like the API, and usage these logs do not hold (claude.ai chat, Claude Code's background calls) draws on the same limits. Several agents on one subscription share the same windows, so one command shows all of them how close the next hit is.

Only tokencur limits looks for limit hits (a fast filter over the logs, about a second on a 1 GB history), so other commands stay as fast as before (ADR 0014).

Money concepts (read before quoting numbers)

tokencur's headline figures are not a bill. Three money concepts, kept deliberately apart:

• Actual outlay — the subscription fees really paid, declared plan by plan in subscriptions.json with the days each plan was active, and provider bills imported as billed charges (see Billed cost). The only real money here.
• Usage value (showback) — what the same usage would cost at API list prices. Subscriptions don't bill per token, so tokencur values the usage instead of pretending to bill it.
• Counterfactuals — cost avoided by provider caching, and what-if right-sizing. Properties of the workload, not actions taken. Under a flat subscription, right-sizing buys rate-limit headroom, not dollars.

Divide value by outlay and you get the observatory's headline metric: subscription leverage — how many times over the fees pay for themselves.

A plan counts only while it was active. from is its first day and until the first day it no longer applied, like a billing period; an upgrade is two plans:

{"plans": {"Claude Code": [
  {"monthly_usd": 20, "from": "2026-09-24", "until": "2026-10-02"},
  {"monthly_usd": 100, "from": "2026-10-02"}
]}}

A flat monthly_usd fee per service still works and counts across the whole window.

Observatory

A public, static snapshot of this repository's own AI spend — the FOCUS dataset rendered as a dashboard: https://pach-boop.github.io/tokencur/observatory/

tokencur observatory   # regenerates docs/observatory/

The snapshot publishes aggregates only (day × service, model and token-bucket totals) — workspace names, session ids and message content never enter the output, and the page is fully self-contained (no external requests). Like the pricing snapshot, it is committed deliberately: the site updates when the maintainer decides, not on a schedule.

Price card

The public list rates tokencur values usage against, and a dated change log built from the git history of the pricing snapshot: https://pach-boop.github.io/tokencur/prices/

tokencur prices   # regenerates docs/prices/

A daily price-watch action refreshes the vendored snapshot and commits whenever it changes, with a message that names what moved (rate moves, models added, models retired upstream and kept at their last rate), then regenerates this page from that history.

Roadmap

Limitations (honest)

• Usage comes from three coding agents' local logs (Claude Code, Codex CLI, Kimi Code); billed cost from RunPod exports. The Anthropic and OpenAI admin cost APIs are next and wait on real response samples.
• The ledger can only keep usage it has seen. Run tokencur more often than Claude Code's cleanupPeriodDays, or raise that setting in ~/.claude/settings.json.
• Codex calls are counted when the session's running total moves; per session, the counted calls reconcile exactly with Codex's own running total (all 94 of the maintainer's rollouts). A forked session that re-copies earlier calls counts them once, in a scan and in the ledger, even after the original log is gone: a copy is recognized by its timestamp and raw usage, so two distinct calls identical to the millisecond would also count once (never observed).
• Costs are list-price showback, not invoices. Subscription plans bill differently.
• CI results come from GitHub Actions only, one named workflow per capture, and a run's conclusion covers all its jobs: a failed run may have failed lint, not tests.
• Claude Code makes calls it never writes to its transcripts: internal calls to smaller models for web search, web fetch and session titles, and some main-model calls such as compaction. Its own end-of-session counters show tokencur's Claude Code value is a slight underestimate: on the maintainer's sessions, tokencur sees 93.6% of the cost Claude Code counted. tokencur doctor measures this on your own logs.
• Usage value per commit counts commits on the checked-out branch, and a commit says nothing about size or quality. Usage logged before tokencur 0.3 kept no working directory; it is attributed only if its log is still on disk.
• Rates are point-in-time: each call is valued at the list rate in force on its UTC day (ADR 0008). Rate history starts with the snapshot on 2026-07-06, so earlier usage is valued at the first rate observed, and a move is dated by the day the price-watch bot saw it. Curated Anthropic rates have no history yet because none has moved; a test fails on any curated rate that changes without a dated history entry or a declared correction (ADR 0011).
• Older log formats don't break down cache writes by TTL; totals are attributed to the 5-minute tier (slight underestimate), documented in the parser.
• Daily buckets use the UTC dates recorded in the logs; a late-night local session can land on the next UTC day.
• Targets FOCUS 1.2: the newest spec version the Foundation's focus-validator can check (2.2.1 ships 1.2 rules only), so newer spec versions will follow the validator. The export passes it in CI and is cross-checked against the Foundation's official sample data (which targets FOCUS 1.0; the tests assert convention compatibility, not column equality).

Contributing

Small, reviewed changes are welcome. CONTRIBUTING.md has the ground rules (metadata only, unpriced is never $0, no runtime dependencies) and how to add a usage source. Changes land through pull requests with green CI; Dependabot, CodeQL and OpenSSF Scorecard run on the repository. Security reports go through SECURITY.md, never a public issue.

Transparency

Built with AI assistance (Claude). Policy: nothing is merged that the maintainer does not fully understand and stand behind.

License

MIT