kvtop

A small, read-only Rust CLI for inspecting KV-cache metrics from Prometheus text snapshots.

Version 0.1.0 reads a UTF-8 file or stdin, selects KV-related metrics, and emits a terminal table or JSON. It is useful with a serving engine's existing metrics endpoint; it does not attach to CUDA or modify a running engine. The Rust executable has no third-party crate dependencies.

Use

curl -fsS http://localhost:8000/metrics | kvtop
kvtop --file snapshot.prom
kvtop --file snapshot.prom --json
kvtop --file snapshot.prom --match prefix_cache

Default name filters cover kv_cache, prefix_cache, kv_block, kv_transfer, cached_tokens, prompt_tokens_cached, and prefill_kv_computed. --match replaces those filters and can be repeated. The input limit is 16 MiB. Empty or missing measurements return an error instead of zero.

Values retain the producer's units and labels. In particular, vLLM's kv_cache_usage_perc uses 1 for 100%, and prefix-cache counters count tokens, not requests. Samples from different workers are kept separate. This tool does not infer physical bytes from generic GPU usage, derive interval hit rates from cumulative counters, or claim a causal performance saving.

JSON has schema_version: 1 and a samples array. Each sample contains name, a raw Prometheus labels string, value, and raw_value. Nonfinite values have JSON null plus their explicit raw_value; very large numeric values may lose precision in value, so use raw_value when exact representation matters. Timestamps and exemplars are not included in this first version. This is a metric-snapshot inspector, not a general Prometheus validator.

Build and test

cargo test
cargo build --release
cargo run -- --file examples/vllm.prom --json

The fixture is synthetic. An actual engine integration and real GPU performance measurements have not been validated by this release preparation.

Distribution

The package is configured for Cargo and for a Python wheel containing the same Rust executable, using Maturin's bin mode. Publishing preparation is not evidence that either registry currently hosts this project. See PUBLISH.md in the source delivery for the commands and publication status.

Existing command with the same name

This is an independent project, unaffiliated with ovg-project/kvcached, which already distributes a Python command named kvtop. Their command inspects kvcached shared-memory records; this program reads Prometheus snapshots. Installing both commands into one environment may overwrite or shadow one entrypoint. Use a separate virtual environment or an explicit executable path. This project does not replace, patch, or depend on kvcached.

License

MIT.