tokenwatt-check

Is the GPU you rented performing like it should?

GPU listings tell you the model — "H100 SXM, verified, 99.5% reliability" — not the card you get. When we benchmarked four rented H100s, two had their power limit lowered by the host and one ran at 787 MHz instead of ~1,900 under load: every token took 2.9× longer. Nothing on the listing showed it.

tokenwatt-check finds that in about two minutes:

pip install tokenwatt-check torch    # torch is usually already there on GPU images
tokenwatt-check

  OVERALL: ❌ FAIL

  ⚠️  power limit            GPU power limit is 500 W of a possible 700 W (71%).
  ❌ clocks under load      SM clock during the matmul averaged 560 MHz — 41% of a healthy H100 (1360 MHz).
                            → Far below normal. Possible causes: aggressive power cap, poor cooling or
                              power delivery, a degraded card. Ask for another machine.
  ❌ BF16 matmul            280 TFLOPS — 41% of the healthy H100 reference (680 TFLOPS).
  ❌ memory copy bandwidth  2.2 TB/s — 73% of the healthy H100 reference (3.0 TB/s).
  ✅ PCIe link              Gen5 x16.
  ✅ disk                   Sequential write 1,616 MB/s.
  ✅ download               2,100 Mbit/s from Hugging Face, one stream (~6.3 min per 100 GB of weights).

(Illustrative output modeled on a real under-performing node — see src/tokenwatt_check/demo.py. Try tokenwatt-check --demo throttled-h100, capped-h100 or healthy-h100 without a GPU.)

What it checks

Results: PASS, WARN or FAIL per check with a reason and what to do, plus a JSON file with every reading (--json path) and an optional Markdown report (--md path). Exit code 2 on FAIL, for scripts.

Options

tokenwatt-check --gpu 1            # another GPU
tokenwatt-check --seconds 60       # longer load test
tokenwatt-check --no-load          # inventory, disk and network only (no PyTorch needed)
tokenwatt-check --no-net --no-disk

Reference values

Compute, bandwidth and clock-under-load are compared with a healthy card of the same model (src/tokenwatt_check/verdict.py):

A dense matmul pulls a healthy card to its power limit, so its clock sits well below the maximum — a healthy H100 holds about 69% of max clock in this test. That's why the clock check compares against a healthy card rather than the maximum. If a healthy card scores low on your machine, please open an issue with your JSON so we can improve the references.

Privacy

Everything runs locally. Nothing is sent anywhere except the 20-second download test from huggingface.co.

Supported

NVIDIA GPUs with nvidia-smi (H100, H200, B200, A100, L40S, RTX 4090/5090 and others). AMD Instinct support is planned.

About

Built by TokenWatt, which publishes vendor-neutral measurements of LLM inference — SLO goodput, tokens per joule and cost per million tokens — on NVIDIA and AMD. MIT licensed.