hmft

Estimate how long an LLM call will take before you send it, and get a live ETA while it streams. Runs entirely on your machine: no prompt or response text ever leaves the process.

pip install hmft                      # from PyPI
pip install "hmft[openai,anthropic]"  # with the SDK wrappers
pip install -e ".[dev]"               # from a clone, with the test extras

Quickstart

No API key, no network call: the estimate is computed locally from the prompt.

import hmft

est = hmft.estimate(
    [{"role": "user", "content": "Explain how bicycles stay upright in two paragraphs."}],
    model="gpt-4.1-mini", provider="openai",
)
print(est)                                      # ~2.3s (p90 4.6s): ttft 0.69s, ~220 tokens [measured]
print(est.total_s.p50, est.total_s.p90)         # 2.27 4.58
print(est.output_tokens.p50, est.output_tokens.p90)  # 220 352

Every estimate is a band, not a point: p50 is the typical case, p90 is the slow-but-likely case you should plan around.

Accuracy on held-out prompts

186 rows from prompt sets A to D, four models, re-scored under the current code with scripts/compare_runs.py. "Coverage" is the share of rows whose actual landed at or under the estimated p90; the target is about 90%.

What "held out" means here, precisely. These 186 rows are held out from v2 constant fitting: compare_runs.py marks sets A to D held-out and fits only on the later sets (E, F). They are not untouched by v1. Set A is the v1 calibration set, and four v1.1 constants were fitted or corrected using rows that are in this table — TOKENS_PER_LIST_ITEM (set B actuals), TOKENS_PER_ONE_LINE_ITEM_P50/P90 (A, B and C), the spoken-duration cue 490 + 59 × minutes (C and D), and the code_large band (the long_code actuals in A, B and C). Read the table below as a standing regression score, not as a clean out-of-sample result. The cleanest out-of-sample evidence is set E, measured on different subjects, where the duration cue covered 29 of 30 rows.

† gpt-5@low passes neither figure, despite the numbers, because all three of its table constants are wrong. The row assumes 1,312 hidden reasoning tokens, where the split runs measured medians of 320 (set E) and 96 (set F); a 9.5 s first token, where the runs measured per-run medians of 4.1 to 7.2 s; and 123 tokens/s decode, where the runs measured 34.9 to 108.6 tokens/s (per-run medians 58 to 81). So hmft over-estimates time for gpt-5@low (set F median actual/estimate 0.55) and under-estimates visible output on long-form prompts (set E 1.27 and 1.34). The 100% coverage above is those two errors cancelling, not a pass. The rebuild ships in 0.3.0. See RESULTS.md.

All four time-coverage breaches across the 140 non-reasoning rows were caused by time to first token, not by the length heuristics; the token estimate was inside p90 in every one.

Supported models

measured means the throughput row comes from a real benchmark run and the model has held-out coverage above. experimental means the throughput row is real but the estimate has not been scored against held-out prompts, or was scored and did not pass. not measured means hmft refuses to invent a number: you get an output-token band and basis="unmeasured", never a fabricated time.

A bare gpt-5 call with no effort set maps to the gpt-5@medium row, the provider default, and basis_key says so. OpenRouter has no provider-wide fallback row, because it routes across model families; an unbenchmarked slug is unmeasured.

Privacy

No prompt text and no response text is ever stored, logged, printed or transmitted. Nothing is uploaded anywhere, by any code path, in v1. There is no server.

After each wrapped call hmft appends one JSON line to ~/.hmft/telemetry.jsonl, created 0600, so you can score your own accuracy offline. The row is an allow-list of integers, floats and short enum words — exactly these 24 fields and nothing else:

ts, hmft_version, model, provider, prompt_tokens, output_tokens, ttft_ms, total_ms, est_output_p50, est_output_p90, est_ttft_p50_s, est_ttft_p90_s, est_total_p50_s, est_total_p90_s, basis, reasoning_effort, effort_source, reasoning_tokens, finish_reason, visible_chunks, first_second_visible_chunks, last_visible_ms, first_event_ms, max_gap_ms.

No prompt, no response, no request ids, no user ids. reasoning_effort, effort_source, basis and finish_reason are validated against fixed sets, so free text is dropped rather than written. The write is enforced by an assertion against the allow-list in hmft/telemetry.py, and tests/test_telemetry.py checks it.

Opt out with one environment variable:

HMFT_TELEMETRY=off                     # write nothing
HMFT_TELEMETRY=/path/to/file.jsonl     # or move it somewhere else

The name

HMFT is short for "how much fucking time" — the question you actually ask when a model has been streaming for forty seconds and you have no idea whether it is nearly done or has barely started. The package, the import and the docs are all just hmft; the long form lives here and nowhere else.

How estimates work

total_time ≈ TTFT(prompt_tokens, model) + output_tokens / tokens_per_second(model)

1. Output length comes from cues in the prompt (hmft/length.py), in priority order: an explicit count ("in one sentence", "about 500 words", "8 questions", "one per line"), a stated speaking duration ("a 10-minute keynote" → 490 + 59 × minutes tokens, validated 2 to 45 minutes), a forced format (JSON mode, forced tool call, max_tokens), and finally a task type guessed from keywords (code, list, JSON, yes/no, translate, summarise, essay, explain, chat), each with its own base band.
2. TTFT and tokens/second are measured constants, per model and provider, in hmft/data/throughput.json. They are only ever written by scripts/benchmark.py from real runs — never estimated, never hand-edited. Unmeasured rows stay null and the estimator reports basis="unmeasured" instead of guessing.
3. The band comes from both. The p50 time uses the throughput p50; the p90 time uses the throughput p10, because for a rate the slow side is the low percentile. The length heuristic contributes its own p50/p90, and the two compose into total_s.

Measured throughput, as of the updated_at in the table:

On reasoning models the TTFT column absorbs hidden thinking time, which is why it is large and why those rows are experimental.

Streaming ETA

from openai import OpenAI
import hmft

client = hmft.wrap_openai(OpenAI())
stream = client.chat.completions.create(
    model="gpt-4.1-mini", stream=True,
    messages=[{"role": "user", "content": "Explain how bicycles stay upright in two paragraphs."}],
)
print(stream.estimate)             # pre-call band
for chunk in stream:
    print(stream.eta(), end="\r")  # live: blends the observed rate in after a few tokens
print(stream.result)               # actual vs estimated

hmft.wrap_anthropic(Anthropic()) is the same shape around client.messages.stream(...). Wrappers are sync-only in v1; async is a v1.x item.

Filling the throughput table

export OPENAI_API_KEY=...       # and/or ANTHROPIC_API_KEY, OPENROUTER_API_KEY
python scripts/benchmark.py --all --dry-run    # print the plan, make no calls
python scripts/benchmark.py --provider anthropic --models claude-haiku-4-5 --runs 3

Each run sends two synthetic prompts and records only token counts and timings. name@effort (for example gpt-5@low) sets reasoning_effort and gets its own row.

Known limits

• Output-length heuristics are rules, not a model. Expect the p50 to be off by 2x on open-ended prompts; the p90 band is what to plan around.
• Reasoning models spend most of their wall time on hidden tokens the heuristics cannot see.
• Non-OpenAI token counts use the o200k tokenizer as a proxy, typically within 10 to 20%.
• Image and tool-result content blocks are not counted.
• Talk-duration cues are validated 2 to 45 minutes; above 45 is extrapolation. The p90 held on 29 of 30 measured talks, but the p50 is biased per model: gpt-4.1-mini writes far less than the estimate on long talks, Claude models somewhat more.
• Provider latency drifts hour to hour; measured_at tells you how stale a row is.

Tests

pip install -e ".[dev]"
python -m pytest

All tests use synthetic prompts and fake clients, and the suite makes no network calls — tests/test_no_network.py enforces that. See CONTRIBUTING.md for the held-out rule and the version-bump rule. BASELINE.md is the frozen v1 reference and RESULTS.md records every measurement since.

License

Apache-2.0. See LICENSE.