cost-per-task

Measure what an AI agent costs per task it completes correctly, not per million tokens. cost-per-task records the tokens each model call used, prices them from dated tables, and reports cost per attempt, cost per solved task and risk-adjusted cost per task, with the statistics needed to trust the figures. Built by OptimNow, implementing the measurement framework published by DoiT.

Get started in 30 seconds

pip install cost-per-task

Python 3.11 or later, no runtime dependencies, dated price tables included. Then pick the way in that matches how you work:

Your situation	What to run
	cpt sessions list, fill in the sheet it writes, then cpt sessions import. Measures the sessions already on your computer from their transcripts, with no proxy, no API key and no extra spend. Guide
	cpt run --task-id issue-142 -- <the command that starts your agent>. A local proxy records every Anthropic, OpenAI or OpenRouter call the agent makes, streaming included, with no change to the agent.
	cpt import langfuse observations.json or cpt import litellm spend_logs.jsonl, to reuse usage you already log.
	pip install "cost-per-task[mcp]", then cpt mcp, so an AI assistant or the OptimNow AI ROI Calculator can read the report.

Every way in ends the same way: label the outcomes, then read the report.

cpt label pass --task issue-142
cpt report --cleanup-cost 25

cpt sessions and the bundled price tables need version 0.5.0 or later. To work from a copy of this repository instead, run pip install -e . in its folder. For a first run, the testing guide compares two models step by step and explains every line of the report.

Why cost per task

Model vendors price tokens. Businesses buy outcomes: a merged pull request, a resolved ticket, a correctly filed invoice. For an agent that makes dozens of model calls, retries when it fails and sometimes returns a wrong answer that looks right, the two units drift far apart:

• A cheaper model that fails more often can cost more per solved task, because the failed attempts are paid for too.
• Agent runs are heavy-tailed. The same task can cost several times more on a bad run, so the average alone understates what you will budget.
• An accepted wrong answer is not free. Someone cleans it up later, and that cleanup belongs in the price of the task.

DoiT's paper Cost Per Task, Not Cost Per Token: A Measurement Framework for the Real Economics of Claude, OpenAI and Grok (13 August 2026) sets out the formulas that turn token counts into those business numbers. Logging usage per call was already solved by LiteLLM, Helicone, Langfuse and OpenTelemetry. The missing half was attempts, outcomes, the division by the success rate and the risk term. This tool supplies that half, for any model.

Who it is for: FinOps and finance teams who need a defensible cost per unit of AI work, engineers comparing two models on a real workload, and anyone publishing agent cost figures who wants them to carry the disclosures that make them checkable. If you can run a command in a terminal, you can use it.

What a report looks like

A real run: a Claude Code session answering a trivial prompt on Haiku, twice, with one of the two answers labelled as a leak to show the risk term at work.

group: claude-haiku-4-5-20251001
  attempts 2 over 1 tasks; labelled 2 (pass 2, fail 0, leaked 1)
  attempt cost C: mean 0.0549 USD, P90 0.0557 USD, total 0.1099 USD
  success rate p: 1.000 (Wilson 95%: 0.342 to 1.000)
  CPT_solved = E[C] / p: 0.0549 USD (bootstrap 95%: 0.0549 to 0.0549, 10000 resamples)
  cost per task attempted (failures included): 0.1099 USD
  capped retries N=2: p_N 1.000
  pass^2: 1.000
  leak rate L: 0.500
  CPT_risk = CPT_solved + L x K: 12.5549 USD (K = 25.0000 USD)
  cache hit rate: 0.0%

disclosure checklist
  model versions: claude-haiku-4-5-20251001
  prices: as of 2026-08-27 in USD; source: OptimNow AI Pricing Hub, cross-checked against the vendor price list
  harness: Claude Code 2.1, -p mode
  ...

Two things this run shows that a per-token view hides. The question and its answer came to 68 tokens, yet 99% of the 5.5 cents went on Claude Code writing its 43,000-token system context into the prompt cache. And with one answer in two flagged as a leak and a $25 cleanup cost, the risk-adjusted cost per task is $12.55, two hundred times the raw cost.

What you get

Per model and per task type, the paper's estimators and what each one tells you:

Every report ends with the paper's disclosure checklist: model versions, prices with dates and source, harness, cache hit rate, effort settings, sample size and k, the intervals used, leak rate, the cleanup cost assumed, and K* for comparisons. With these attached, a cost figure becomes a measurement someone else can check.

To compare two models directly:

cpt compare claude-haiku-4-5 claude-sonnet-5 --cleanup-cost 25

How it works

Three ways in, one log. Whatever the source, each model call becomes one line in a JSONL log: token counts by class, model, provider, task and attempt ids, tool names, latency and the effort level requested. Outcomes go to a separate labels file, so the usage log is never rewritten.

• Local proxy. cpt run starts a small web server on your machine and points the agent at it through ANTHROPIC_BASE_URL and OPENAI_BASE_URL, the standard variables every SDK honours. Requests reach the real API unchanged and responses come back unchanged, streaming included. On the way through, the proxy copies the vendor's own token counts.
• Claude Code and Cowork transcripts. Both products keep a transcript of each session on disk, with the usage block Anthropic returned for every response. cpt sessions reads them, counts each response once, folds sub-agent work into its parent session and gives you a spreadsheet to label.
• Exports. Langfuse observations and LiteLLM spend logs, as CSV, JSON or JSONL.

Counted, never estimated. Token counts come from the vendor's response, never from a local tokenizer.

Never logged. API keys, headers, prompts and completions. Prompts often hold client data and have no place in a metrics file. A test checks this on every commit.

Attempts that mix models, such as an agent using a small model for side calls, are attributed to the model carrying the largest share of the cost.

What it works with

Native adapters for Amazon Bedrock, Google Vertex AI and xAI are on the roadmap. Each is a small file, because only the shape of the usage block differs between vendors. Statistics, labels, report and comparison are model-agnostic already.

Commands

Useful options on report and compare: --prices to use your own tables (repeatable), --leak-rate to override the measured L, --retry-cap N, --k, --seed for reproducible bootstrap intervals, --task-type to filter and --by-model-only to ignore task types.

Prices

Prices live in dated JSON tables, one per vendor, per model and token class, per million tokens. A table without an as_of date is refused, because the disclosure checklist requires prices with dates. The Anthropic and OpenAI tables ship with the tool and apply when you do not pass --prices; pass your own table to use negotiated rates.

{
  "currency": "USD",
  "as_of": "2026-09-13",
  "source": "OptimNow AI Pricing Hub, cross-checked against platform.claude.com",
  "models": {
    "claude-sonnet-5": {
      "input_per_mtok": 2.0,
      "cache_read_per_mtok": 0.2,
      "cache_write_5m_per_mtok": 2.5,
      "cache_write_1h_per_mtok": 4.0,
      "output_per_mtok": 10.0,
      "reasoning_per_mtok": null
    }
  }
}

reasoning_per_mtok: null means reasoning is billed at the output rate, which is how both vendors price today. Model ids reported with a date suffix (claude-haiku-4-5-20251001) or a gateway prefix (anthropic/claude-haiku-4.5) match the table automatically.

Keeping prices current. The OptimNow AI Pricing Hub, which serves the OptimToken catalogue of 250+ models refreshed daily, is the upstream:

cpt prices refresh --provider anthropic            # shows what would change
cpt prices refresh --provider anthropic --write    # accepts it

The default run lists new, removed and repriced models with the catalogue date, warns about models the hub cannot price fully and flags cache-read prices that look wrong. Nothing is written without --write, so an upstream feed error never lands unseen.

Known limitation. One rate per token class. Long-context tiers (Anthropic above 200K input tokens, OpenAI above 272K) and batch discounts are not modelled. If your attempts cross those thresholds the report understates cost, and it states which prices it applied.

For AI assistants and other tools

cpt report --json and cpt compare --json emit the summaries as JSON. The same figures are available over MCP, so an assistant can answer "what does a solved ticket cost us on Sonnet, with the interval?" from your own log, and the OptimNow AI ROI Calculator can use CPT_risk as its cost denominator instead of a per-token guess:

pip install "cost-per-task[mcp]"
cpt mcp

Tools: cpt_report, cpt_compare and cpt_risk_denominator (CPT_risk for one model with sample size, intervals and price date). The mcp extra is the only optional dependency; the core has none.

Directory structure

cost-per-task/
├── README.md                 <- This file
├── LICENSE                   <- MIT
├── CHANGELOG.md              <- Release notes
├── CLAUDE.md                 <- Project context and hard rules for AI assistants
├── assets/                   <- Diagrams used in this README
├── docs/                     <- Testing guide, Claude Code and Cowork guide
├── examples/                 <- Scripts and tasks used by the testing guide
├── prices/                   <- Dated price tables, shipped inside the package too
├── src/cost_per_task/
│   ├── proxy.py              <- The local capture proxy
│   ├── providers/            <- Per-vendor usage extraction (anthropic.py, openai.py)
│   ├── importers/            <- Langfuse, LiteLLM, Claude Code and Cowork sessions
│   ├── schema.py             <- The JSONL record, aligned with OpenTelemetry GenAI
│   ├── labels.py             <- Pass, fail and leak labels, CSV import
│   ├── pricing.py            <- Dated tables and per-call cost
│   ├── prices_hub.py         <- Price refresh from the OptimNow AI Pricing Hub
│   ├── stats.py              <- Wilson, bootstrap, percentiles, p_N, pass^k
│   ├── metrics.py            <- Attempts, groups, CPT_solved, CPT_risk, K*
│   ├── report.py             <- Text and JSON report, disclosure checklist
│   ├── analysis.py           <- Shared loader for the CLI and the MCP server
│   ├── mcp_server.py         <- MCP tools (optional extra)
│   └── cli.py                <- The cpt command
├── tests/                    <- pytest; proxy tests run against fake vendor servers
└── .github/workflows/        <- CI, and the PyPI release on version tags

Design principles

• Token counts come from the vendor, never from a local tokenizer. The paper's first rule, and the difference between a measurement and an estimate.
• No number without a date and a source. Price tables carry as_of and source, and the report repeats them. A table without a date will not load.
• Never invent a price. The Pricing Hub refresh skips models it cannot fully price and flags anomalies rather than guessing; a person decides with --write.
• Never log content. Keys, headers, prompts and completions stay out of the log, enforced by a test.
• Report the spread, not only the mean. P90, Wilson and bootstrap intervals are always shown, so a small sample reads as a wide interval instead of hiding.
• The log is append-only. Outcomes and corrections live in the labels file.
• Nothing to install around it. Standard library only. The MCP server is the one optional extra, and it is imported only when asked for.

Status

Version 0.5.0 is on PyPI. What has been checked, and how:

Roadmap: live validation of OpenAI and OpenRouter, importers checked on real exports, Amazon Bedrock, Google Vertex AI and xAI adapters, long-context price tiers.

Contributing

The most valuable contributions are real logs and real exports: "we ran this through the proxy and the invoice said X" is worth more than any feature. Provider adapters, corrections to pricing rules with a source, and adversarial review of the statistics are also welcome. Open an issue first for anything structural.

About OptimNow

OptimNow is a boutique FinOps consultancy helping organisations connect cloud and AI spend to measurable business value. Based in France with European reach.

• Website: optimnow.io
• LinkedIn: OptimNow
• GitHub: github.com/OptimNow

Open-source tools built by OptimNow:

Acknowledgements

The measurement model implemented here is the work of DoiT. That covers the token classes, cost per solved task, the capped-retry variant, the risk-adjusted cost with leak rate and cleanup cost, the break-even cleanup cost K*, the choice of Wilson and bootstrap intervals, pass^k and the disclosure checklist. It is published as Cost Per Task, Not Cost Per Token: A Measurement Framework for the Real Economics of Claude, OpenAI and Grok (DoiT Research, 13 August 2026), under CC BY 4.0: https://www.doit.com/research/economics-of-claude-openai-and-grok. Read the paper for the reasoning behind each estimator; this repository makes them runnable, and the formula names in the report (C_attempt, CPT_solved, CPT_risk, K*) follow the paper so the two can be read side by side.

Prices come from the OptimNow AI Pricing Hub and are cross-checked against the vendors' published price lists, with the date of each check recorded in the tables.

This tool is independently maintained by OptimNow and is not affiliated with or endorsed by DoiT, Anthropic, OpenAI or OpenRouter. Any implementation errors are ours.

License

Released under the MIT License, copyright 2026 OptimNow. You may use, modify and redistribute the software, including commercially, provided the copyright and licence notice are kept. The methodology remains DoiT's, published under CC BY 4.0: please cite their paper when you publish figures produced with this tool.