hotcoco 0.3.0

Drop-in pycocotools replacement — up to 23x faster COCO evaluation in pure Rust

hotcoco

Fast enough for every epoch, lean enough for every dataset. A drop-in replacement for pycocotools that doesn't become the bottleneck — in your training loop or at foundation model scale. Up to 23× faster on standard COCO, 39× faster on Objects365, and fits comfortably in memory where alternatives run out.

Available as a Python package, CLI tool, and Rust library. Pure Rust — no Cython, no C compiler, no Microsoft Build Tools. Prebuilt wheels for Linux, macOS, and Windows.

Beyond raw speed, hotcoco ships a diagnostic toolkit that pycocotools and faster-coco-eval don't have: TIDE error breakdown, cross-category confusion matrix, per-category AP, F-scores, sliced evaluation, dataset healthcheck, and publication-quality plots with a one-call PDF report. Same pip install, no extra config.

Documentation | Changelog | Roadmap

Performance

Benchmarked on COCO val2017 (5,000 images, 36,781 synthetic detections), Apple M1 MacBook Air:

Eval Type	pycocotools	faster-coco-eval	hotcoco
bbox	9.46s	2.45s (3.9×)	0.41s (23.0×)
segm	9.16s	4.36s (2.1×)	0.49s (18.6×)
keypoints	2.62s	1.78s (1.5×)	0.21s (12.7×)

Speedups in parentheses are vs pycocotools. Results verified against pycocotools on COCO val2017 with a 10,000+ case parity test suite — your AP scores won't change.

At scale (Objects365 val — 80k images, 365 categories, 1.2M detections), hotcoco completes in 18s vs 721s for pycocotools (39×) and 251s for faster-coco-eval (14×) — while using half the memory. See the full benchmarks.

Get started

pip install hotcoco

No Cython, no C compiler, no Microsoft Build Tools. Prebuilt wheels for Linux, macOS, and Windows.

Already using pycocotools? One line:

from hotcoco import init_as_pycocotools
init_as_pycocotools()

Or use it directly — the API is identical:

from hotcoco import COCO, COCOeval

coco_gt = COCO("instances_val2017.json")
coco_dt = coco_gt.load_res("detections.json")

ev = COCOeval(coco_gt, coco_dt, "bbox")
ev.run()

What's included

• COCO, LVIS & Open Images evaluation — bbox, segmentation, and keypoints; all standard metrics plus LVIS federated eval (APr/APc/APf) and Open Images hierarchy-aware eval (group-of matching, GT expansion). See the evaluation guide.
• TIDE error analysis — breaks down every FP and FN into six error types so you know why your model falls short, not just by how much. See TIDE errors.
• Confusion matrix — cross-category matching with per-class breakdowns. See confusion matrix.
• F-scores — F-beta averaging over precision/recall curves, analogous to mAP. See F-scores.
• Plotting — publication-quality PR curves, per-category AP, confusion matrices, and TIDE error breakdowns. Three built-in themes (warm-slate, scientific-blue, ember) with paper_mode for LaTeX/PowerPoint embedding. report() generates a single-page PDF summary. pip install hotcoco[plot]. See plotting.
• Sliced evaluation — re-accumulate metrics for named image subsets (indoor/outdoor, day/night) without recomputing IoU. See sliced evaluation.
• Dataset healthcheck — 4-layer validation (structural, quality, distribution, GT/DT compatibility) catches duplicate IDs, degenerate bboxes, category imbalance, and more. See healthcheck.
• Format conversion — COCO ↔ YOLO in either direction, from Python or the CLI. See format conversion.
• PyTorch integrations — CocoDetection and CocoEvaluator drop-in replacements for torchvision's detection classes; no torchvision or pycocotools dependency required. See PyTorch integration.
• Experiment tracker integration — get_results(prefix="val/bbox", per_class=True) returns a flat dict ready for W&B, MLflow, or any logger. See logging metrics.
• Python CLI (coco) — included with pip install hotcoco; eval, healthcheck, stats, filter, merge, split, sample, and convert subcommands. See CLI reference.
• Rust CLI (coco-eval) — lightweight eval-only binary; cargo install hotcoco-cli. See CLI reference.
• Rust library — use hotcoco directly in your Rust projects via cargo add hotcoco. See Rust API.

See the documentation for full API reference and examples.

Contributing

Contributions are welcome. The core library is pure Rust in crates/hotcoco/ — if you're new to Rust but comfortable with Python and the COCO spec, the PyO3 bindings in crates/hotcoco-pyo3/ are a gentler entry point.

Before submitting a PR, run the pre-commit checks locally:

cargo fmt --all
cargo clippy --workspace --all-targets -- -D warnings
cargo test

Parity with pycocotools is a hard requirement — if your change touches evaluation logic, verify metrics haven't shifted with just parity (see CONTRIBUTING.md).

License

MIT