win-llmtop

A live, read-only terminal panel for llama.cpp's llama-server on Windows — see where your VRAM, shared GPU memory and system RAM go while a model runs.

What it shows · Who it is for · Quick start · Install · Requirements · Get more out of it · Common options · Documentation

What it shows

win-llmtop is strictly read-only: it reads the server's HTTP API, log files and Windows performance counters, and never changes the server.

• Dedicated and shared GPU memory per process against the DXGI OS budget, with alerts when Windows demotes VRAM into system RAM.
• System RAM, commit charge and the page file — working sets by consumer, private commit headroom and disk page-ins/page-outs.
• Live in-flight request progress — active phase (PREFILL or DECODE), prompt batch progress, ETA, live tokens/s and KV context fill.
• Finished request history and speed — prefill and decode throughput compared across similar prompt sizes and context depths, draft acceptance and prefix cache reuse.
• Model memory map and tensor placement — weights, KV cache, recurrent state and compute buffers derived from the GGUF header and checked against llama-fit-params, plus per-layer GPU/CPU placement.
• Router and threshold events — model loads, queueing, evictions and sustained memory alerts persisted across restarts.

Who it is for

Built by Aaron Kane (@insane-kane) for Windows users running llama.cpp's llama-server — either in router mode across multiple presets or serving a single model — on the same machine. It answers whether a preset fits inside Windows' VRAM budget, when shared GPU memory spillover slows decoding down, and where host RAM goes.

Quick start

pipx install win-llmtop
win-llmtop --check
win-llmtop

• pipx install win-llmtop — installs win-llmtop into an isolated environment and puts it on PATH.
• win-llmtop --check — checks Windows counters, the GPU budget, the running llama-server, logs and state directory, and exits 0 when the server is reachable and either GPU telemetry is present or the machine has no GPU adapters (1 otherwise).
• win-llmtop — opens the live six-page terminal panel (1–6 or Tab switch pages, q quits).

Install options at a glance

See the full installation guide (docs/install.md) for commands, upgrades, uninstalls and package verification.

Requirements

• Windows 10 version 1709 or later — provides the per-process GPU Process Memory and GPU Engine performance counters.
• 64-bit Python 3.11 or later — 32-bit Python cannot inspect a 64-bit llama-server process.
• No third-party Python packages — uses the Python standard library only.
• A local llama-server — running in router mode or single-model mode on the same machine.

Get more out of it

win-llmtop works with zero configuration, and unlocks deeper telemetry when llama-server exposes logs and slots:

• Per-model log file with timestamps — pass --log-file and --log-timestamps to llama-server (win-llmtop reads the path from its command line; --log-dir only when it can't) for request history, cache hit rates, draft acceptance and wall-clock times (usage guide).
• Live /slots telemetry — start llama-server with --slots to see live prefill/decode progress, current tokens/s, ETA and context fill (usage guide).
• Router event log — pass --router-log or --log-dir so win-llmtop can tail router model loads, request queueing and preset evictions (usage guide).

Common options

See win-llmtop(1) for the complete reference of every command-line option, environment variable and default.

Documentation

• Documentation index (docs/README.md) — one-line map of all documentation pages.
• Installation (docs/install.md) — every deployment method, offline wheels, provenance verification, 64-bit Python and terminal setup.
• Usage (docs/usage.md) — --check output, the six pages, keyboard controls, llama-server flags, --once, --plan and state files.
• Troubleshooting (docs/troubleshooting.md) — symptom, cause and fix for every --check failure, notice and warning.
• Manual page (docs/llmtop.1.md) — win-llmtop(1) reference: all options, alert rules, data sources and measurement limits.

Contributing, support and security

• Contributing and support — see CONTRIBUTING.md to report bugs or submit pull requests, and SUPPORT.md for usage help.
• Security — see SECURITY.md to report a vulnerability privately through GitHub's private vulnerability reporting.
• Code of conduct — participation is governed by CODE_OF_CONDUCT.md.

Citing

If you use win-llmtop in research or benchmarks, see CITATION.cff for citation metadata.

Licence

GNU General Public License v3.0 or later (GPL-3.0-or-later); see LICENSE.