vllm-doctor 0.2.0

Diagnostic tool for vLLM inference servers

Diagnose vLLM serving issues from /metrics.

vLLM Doctor reads production metrics and turns them into operational findings: what looks wrong, how confident the diagnosis is, and which vLLM knobs are worth checking first.

vllm-doctor --url http://localhost:8000/metrics

vLLM Doctor is not a dashboard replacement. It is a fast diagnostic snapshot for a single server or Prometheus target.

Why not just a dashboard?

Dashboards show metrics. vLLM Doctor explains inference-system behavior.

	Dashboards	vLLM Doctor
Shows raw metrics	✓	✓
Explains what's wrong	✗	✓
Recommends vLLM configs	✗	✓
Requires setup	✓	✗
Works on a single server	✗	✓

Installation

With pip:

pip install vllm-doctor

With uv:

uv tool install vllm-doctor

Quickstart

Direct scrape:

vllm-doctor --url http://localhost:8000/metrics

Prometheus:

vllm-doctor --url http://localhost:9090

Options:

Usage: vllm-doctor [OPTIONS]

Options:
  -u, --url      TEXT         URL to diagnose (vLLM /metrics or Prometheus).  [required]
  -w, --window   TEXT         Time window (e.g. '1h', '30m', 'now').  [default: now]
  -f, --format   [text|json]  Output format.  [default: text]
  -v, --verbose               Show additional diagnostic detail.
  -l, --live     INTEGER      Refresh interval in seconds (e.g. --live 10).
  -c, --config   PATH         Path to config file (default: vllm-doctor.toml).
      --help                  Show this message and exit.

Example output

─────────── vLLM Doctor  ·  Health: CRITICAL  ·  Window: 5m ────────────

╭─ ⚠ Queue pressure  [low confidence] ─────────────────────────────────╮
│   Waiting requests: 7                                                │
│                                                                      │
│   → Add replicas or increase concurrency limits                      │
│   → Inspect autoscaling thresholds                                   │
╰──────────────────────────────────────────────────────────────────────╯
╭─ ✖ KV cache pressure  [high confidence] ─────────────────────────────╮
│   GPU KV cache usage: 94%  ·  Waiting requests: 7                    │
│                                                                      │
│   → Reduce max_num_seqs to limit concurrent sequences                │
│   → Increase gpu_memory_utilization if GPU memory headroom exists    │
╰──────────────────────────────────────────────────────────────────────╯

  Queue Pressure       ⚠ warning     [low]
  KV Cache Pressure    ✖ critical    [high]
  Low Throughput       ✓ ok
  Error Rate           ✓ ok
  High TTFT            ✓ ok

─────────────────────────── Observed Metrics ───────────────────────────

  Requests Running                             12
  Requests Waiting                              7
  GPU Cache Usage        ███████████████████░ 94%
  Generation Tokens/s                        42.0
  TTFT p95 (s)                              3.200
  TPOT p95 (s)                              0.050

Documentation

Read the full documentation: https://aminalaee.github.io/vllm-doctor