llm-rates

Named llm-rates (not llm-prices, its name until 2026-08-21) because PyPI's similarity check rejected llm-prices as too close to the existing llmprices project — collapsed-separator collision, not an exact-name clash. llm-rates was verified free in every form (llm-rates, llmrates, llm_rates, singular llm-rate/llmrate) before adopting it; don't re-litigate the name.

Source is maintained in a private monorepo. Releases are built and published by GitHub Actions through a PyPI Trusted Publisher; provenance attestations are visible on the PyPI file listing.

One vendored model-price catalog + a two-layer overlay (a packaged public layer plus an optional, out-of-wheel private layer) + a shared Python lookup/cost-math module, replacing four independently hand-maintained price tables that were drifting apart. Full origin story and decisions: this repo's sibling checkout plugin/docs/plans/model-catalog-consolidation.md (cross-repo — plugin is a separate git repository from libs, so this is a path reference, not a clickable link).

Why this exists

Four tables held the same vendor facts (plugin/scripts/prices/anthropic.json, plugin/skills/tokenator/scripts/pricing.json, sessions/src/sessmon/pricing.py, factory-bench/runs/model-bench/model-bench-runner.py). One of them silently returned $0.00 for any model it didn't recognise — a lookup bug, not a data bug — and under-reported real spend by over $800 in one 30-day window. This package fixes the lookup logic (never $0 for an unknown model) and gives the data one home.

Files

• catalog.json — vendored snapshot of litellm's model_prices_and_context_window.json, filtered to the providers/models the portfolio actually uses. Vendor list price, per-single-token USD (litellm's native unit). Never hand-edit — regenerate prices via refresh.py --apply; for Anthropic-provider rows, the capability fields (max_input_tokens/max_output_tokens) are instead kept in sync with Anthropic's own live Models API via refresh_context.py --apply (litellm's snapshot is a third-party guess for those fields, not authoritative).
• overlay.json — the public overlay layer, packaged inside the wheel: family-rate fallbacks (claude-opus → tier rate, so an unrecognised new model prices at its family's rate instead of $0), model-ID aliases (dated suffixes → canonical key), retirement/deprecation history (a retired model's row is kept, never deleted, so historical transcripts still reprice correctly), context_overrides for the rare model whose real default-served context genuinely differs from its published maximum (empty as of 2026-08-21 — see "Context-window data" below), and the one public free-preview actual_billing_overrides row (stealth/ox-alpha). It's enough on its own — load_overlay() with no private overlay configured is a fully supported state. Portfolio-internal price facts (free-tier billing keys, per-consumer pin lists) live in an optional private overlay layer, portfolio/overlay.json — outside the wheel, resolved at call time via LLM_RATES_OVERLAY/PORTFOLIO_DATA_ROOT, and merged in by load_overlay(). See portfolio/README.md and the "Overlay: public and private layers" section below.
• llm_rates/ — lookup(model_id) -> PriceRecord + cost math (in __init__.py); catalog.json and overlay.json ship inside this directory as package data so an installed wheel carries them alongside the code. Shared by every Python consumer; the sole PowerShell consumer (tokenator.ps1) can't import Python, so it reads a generated JSON table instead — see generate_tokenator_table.py below and the plan doc's "Known constraint".
• refresh.py — re-pulls the live litellm catalog, re-filters it to the same model set, diffs against the vendored catalog.json, and prints a report. Never applies silently — pass --apply to write. Price source only; does not touch capability fields for Anthropic-provider rows.
• refresh_context.py — re-pulls max_input_tokens/max_tokens for every Anthropic-provider catalog.json row from Anthropic's own live Models API (client.models.retrieve()), diffs against the vendored values, and prints a report. Never applies silently — pass --apply to write. Requires the anthropic package (pip install "llm-rates[refresh]") and a resolvable Anthropic credential (ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, or an ant auth login profile) — never falls back to a guessed value if no credential resolves. Run manually; not wired into any scheduled job. lookup()/cost() never import this module or touch the network — see "Context-window data" below.
• generate_tokenator_table.py — emits a plain JSON price table shaped for tokenator.ps1 (a models map of {input, output, context} per model id, plus flat cache_read_multiplier/cache_write_multiplier and a default row) from catalog.json + overlay.json, so that repo can hold a generated copy instead of a hand-maintained one. Deterministic — the same inputs always produce byte-identical output.

Usage

from llm_rates import lookup, cost, UnknownModelError

record = lookup("claude-sonnet-5")
record.input          # 2.0   ($/MTok)
record.output          # 10.0  ($/MTok)
record.source           # "catalog"
record.context           # 1000000 (context-window token limit, or None)

# A client-side "[1m]" resolvedModel signal (not a vendor id — see
# overlay.json's aliases) resolves to the same price and, as of 2026-08-21,
# the same context — the base id already reports the model's real
# 1,000,000-token window:
lookup("claude-sonnet-5[1m]").context   # 1000000

# One-shot cost for a turn:
usd = cost(
    "claude-opus-4-8",
    input_tokens=12_000,
    output_tokens=800,
    cache_read_tokens=50_000,
)

# An unrecognised model still resolves at its family rate:
lookup("claude-opus-4-9").source   # "overlay-family-fallback"

# A genuinely unknown vendor/model raises instead of returning $0:
try:
    lookup("some-new-vendor/mystery-model")
except UnknownModelError as e:
    ...

lookup() and cost() both accept optional catalog=/overlay= kwargs (already-loaded dicts) — useful for tests, or for a caller that wants to load once and reuse across many lookups instead of re-reading the JSON files each call.

Overlay: public and private layers

overlay.json (this directory) is the public layer and ships inside the wheel — aliases, family fallbacks, retirements, context_overrides, and the one public free-preview actual_billing_overrides row. It resolves every current model family on its own; pip install llm-rates with no further setup is a fully supported state, never a degraded one.

An optional private layer, portfolio/overlay.json (this repo only, not packaged — see portfolio/README.md), carries portfolio-internal price facts: free-tier billing overrides and per-consumer pin lists. load_overlay() (no args) merges the two, private winning per key, once one of two environment variables points at the private file:

1. LLM_RATES_OVERLAY — set to the private overlay's exact path. Must exist and parse if set and non-empty, else OverlayNotFoundError (a typo'd path silently falling back to public-only would recreate the silent-mispricing bug this package exists to kill). Set to "" to mean "explicitly none".
2. PORTFOLIO_DATA_ROOT — if LLM_RATES_OVERLAY is unset, used to look for <root>/config/llm-rates/overlay.json; used if present, silently skipped if not.

With neither variable set, load_overlay() returns the public overlay alone. load_overlay(path) with an explicit path always loads exactly that one file, unmerged — today's meaning, unchanged.

Refreshing the catalog

python refresh.py            # pull live litellm catalog, diff, print report
python refresh.py --apply    # also overwrite catalog.json with the diff

refresh.py always pulls the live GitHub-hosted catalog URL, never the litellm pip package's bundled snapshot — that bundled copy is stale (it was missing Haiku 4.5, Opus 5, Opus 4.8, Fable 5, and Sonnet 5 entirely as of 2026-07-28; see the plan doc's "Two traps found").

Refreshing context-window data

python refresh_context.py            # pull the live Models API, diff, print report
python refresh_context.py --apply    # also overwrite catalog.json with the diff

Context-window limits (max_input_tokens) and output caps (max_output_tokens) for Anthropic-provider rows come from Anthropic's own Models API (client.models.retrieve(model_id)), not litellm's snapshot — litellm is a third-party-maintained guess for these fields, and it was wrong at least once (see "Pricing notes worth knowing" below). lookup() itself never calls this API: it's a refresh-time-only script, exactly like refresh.py, that writes into the vendored catalog.json, so lookup()/cost() stay pure and offline (they price historical transcripts and run behind a PowerShell overlay with no network access). Requires pip install "llm-rates[refresh]" (the anthropic SDK) and a credential the SDK's own resolution chain can find (ANTHROPIC_API_KEY/ANTHROPIC_AUTH_TOKEN/ant auth login) — with none resolvable, the script fails with an actionable message and writes nothing, never a guessed value.

Generating tokenator's pricing table

python generate_tokenator_table.py            # print to stdout
python generate_tokenator_table.py --out FILE  # write to FILE

This is a read-only export — it never touches plugin's checkout. The consumer is plugin/tools/tokenator/pricing.json (tokenator was revived as a plugin tool 2026-09-05, plugin#1006, after living at plugin/skills/tokenator/scripts/ as a skill). That file is the generated table as of 2026-09-05 — regenerate with --out pointed at it whenever catalog.json/overlay.json change. context is passed through from lookup() unchanged — the catalog value refresh_context.py verifies against the Models API. A same-day override to 200,000 for bare claude-fable-5/claude-sonnet-5 was reverted 2026-09-05 after a scan of every local transcript showed those sessions running past 500k; the correct fix for a wrong window is always catalog.json via refresh_context.py. Before emitting anything, the generator proves (via verify_cache_multipliers()) that its flat cache multipliers (0.1x read, 2.0x write of input — tuned to Claude Code's 1h-TTL cache behaviour) match this package's real per-model cache rates for every model tokenator prices; a divergence raises instead of silently drifting.

Pricing notes worth knowing

• claude-sonnet-5 is $2.00/$10.00 per MTok, permanently — not a time-limited introductory rate. It was announced as an introductory price through 2026-08-31, but the vendor cancelled the scheduled 2026-09-01 rise to $3.00/$15.00 (~2026-08-17). See overlay.json's notes.
• Groq OSS models list at $0.075–$0.29/MTok but bill $0.00 on the portfolio's free-tier key — the vendor list price lives in catalog.json, the actual billed rate lives in portfolio/overlay.json's (the private layer's) actual_billing_overrides, picked up automatically once LLM_RATES_OVERLAY/PORTFOLIO_DATA_ROOT points at it. Both prices are correct; they answer different questions. Without the private overlay configured, these models resolve at the vendor's list price via catalog.json — never a silent $0 either way.
• A retired/de-listed model keeps its row. claude-opus-4-1 (retired 2026-08-05) and the Groq-delisted llama-4-scout/qwen3-32b entries stay priced at their historical rate so old transcripts still reprice correctly — deleting a row would silently re-price past usage at whatever family-fallback rate happens to apply now.
• claude-fable-5, claude-opus-5, and claude-sonnet-5 all serve 1,000,000 tokens of context, natively — confirmed by Anthropic's own Models API. A prior overlay.json context_overrides block forced all three to 200,000 (on the mistaken theory that catalog.json's max_input_tokens=1000000 was an extended-context-beta-only ceiling); that override has been removed (corrected 2026-08-21, C8). Only claude-haiku-4-5 among current Claude models genuinely has a 200,000-token window.
• claude-fable-5-1 reads cache at $0.25/MTok (0.025× input), not the 0.1× every earlier Anthropic model uses. Its catalog row carries the explicit cache_read_input_token_cost; lookup() prefers a row's explicit rate over the derived multiplier, so the $1.00 the claude-fable family fallback would derive never applies. Context 1,000,000, max output 128,000, same as Fable 5.

Setup for consumers

pip install llm-rates

Same convention as libs/plan-doc and libs/triad-base — see plugin/CLAUDE.md.

Status

Rollout step 1 of model-catalog-consolidation.md — the library itself, built and tested. Consumer migrations (activity_cost.py, sessmon, model-bench-runner.py, tokenator.ps1) are steps 2–5, tracked separately in the plan doc and out of scope for this package's initial PR.

Testing

pytest llm-rates/tests/ -v