llamastat

nvtop for llama.cpp: a zero-infrastructure terminal dashboard for llama-server inference. Point it at a running server and watch slots, throughput, and context usage live.

Why llamastat?

llama-server tells you what it is doing, but only in raw JSON. llamastat turns that into a live picture:

• Per-slot activity: which slots are processing prompts, which are generating, and how full each context window is.
• Throughput without --metrics: llamastat diffs slot token counters between polls, so you get generation and prompt tokens/s even on servers started without the Prometheus endpoint.
• Zero infrastructure: no Prometheus, no Grafana, no agents. One pip install, one command, works over any HTTP reachable server (including an SSH port-forward to a production box).
• Degrades gracefully: if the server flaps, is still loading the model, or has /slots disabled, llamastat shows what it can instead of crashing.

Sibling project: vllmstat does the same for vLLM.

Install

pip install llamastat

Usage

llamastat                          # local server on the default port
llamastat -u http://host:8080      # remote server
llamastat -u http://host:8080 -i 0.5
llamastat --once --json            # one snapshot as JSON, for scripting
llamastat --mock                   # synthetic data, no server needed

Key bindings

Flags

What it shows

• Header: model alias, quantization, parameter count, file size, context per slot, training context, server build, health.
• Slots: one row per slot with phase (prompt/gen/idle), task id, prompt progress, generated tokens, per-slot tokens/s, and a context-usage bar.
• Throughput: EWMA generation and prompt tokens/s with session peaks and totals, plus a braille history plot.
• Requests: running/peak concurrency, completed tasks, aggregate context in use.
• Server metrics: when the server was started with --metrics, llamastat also shows the server-side Prometheus counters (total tokens, deferred requests, context high-water mark, busy slots per decode). The panel hides itself otherwise.
• Sampling defaults: temperature, top_p, top_k, min_p, max_tokens from /props.

llamastat only issues read-only GET requests (/health, /slots, /props, /v1/models, and optionally /metrics), so it is safe to point at a production server.

Requirements

• Python 3.10+
• A llama.cpp llama-server new enough to expose /slots (enabled by default; some deployments disable it, in which case llamastat falls back to header and metrics panels)

Development

git clone https://github.com/bryanvine/llamastat
cd llamastat
pip install -e ".[dev]"
pytest -q

See CONTRIBUTING.md.

License

Apache-2.0. See LICENSE.