mtop

htop for Ollama — a curses-based TUI that monitors your models, GPU, and the Ollama server (in Docker, Podman or bare-metal) in real time. Zero flicker. Zero dependencies beyond Python 3.10+.

Why?

There are web dashboards, Prometheus exporters, and chat TUIs for Ollama. But there's no terminal monitor — something you SSH into a box and just run, like htop or nvtop, to see what models are loaded, how much VRAM they're eating, and whether the container is healthy.

mtop fills that gap. One file, one command, pure stdlib Python. It auto-detects whether Ollama runs in a Docker/Podman container or as a bare-metal process (systemd or a manual ollama serve) and monitors it either way.

It also watches llama-swap fronting vLLM (--backend llama-swap), with the same host, GPU and runner views. See llama-swap + vLLM. And ComfyUI (--backend comfyui): the models its jobs load, live job progress with ETA, the queue and the model catalog, with --control to cancel, clear and free. See ComfyUI.

Features

• Zero-flicker display — curses double-buffered rendering, no clear + print loops
• Effective inference config — RUNNERS section parses each runner's argv: context, batch, flash attention, KV cache dtype, layer offload. The only place these are observable
• Whole-tree process stats — the server and its per-model ollama runner subprocesses, with PSS accounting where available
• Server config — the OLLAMA_* / CUDA_VISIBLE_DEVICES / HSA_* environment the server was started with, from the container, the process, or the systemd unit; next to RUNNERS this is "what was set" vs "what was negotiated"
• Which model on which card — NVIDIA per-process VRAM via NVML (ctypes, no nvidia-smi fork) joined to the runner PIDs: a GPU column in RUNNERS and a procs: line under each card
• Loaded models — name, VRAM/RAM split, context length, processor type, TTL countdown
• Container health — status indicator (●/✗/○), uptime, CPU & memory with progress bars
• Multi-vendor GPU monitoring — NVIDIA via NVML (nvidia-smi as fallback), AMD and Intel via sysfs, Apple Silicon via ioreg; side by side on the same host; utilization, VRAM/GTT, temperature, power draw; ACPI thermal zones alongside (on the DGX Spark they differ from the GPU sensor)
• AMD without ROCm — telemetry comes from /sys/class/drm/card*/device, so a bare amdgpu driver is enough; rocm-smi is only a fallback
• Jetson / Tegra / NVIDIA Spark — automatic fallback to unified memory via /proc/meminfo
• Non-blocking UI — all I/O (docker, nvidia-smi, HTTP) runs in a background collector thread; the interface stays responsive at 100 ms even when the API hangs, and stale data is flagged
• Interactive — q to quit, +/- to adjust refresh interval, o to toggle raw ollama ps
• Log panel with request stats — tail of the container log or journalctl, with requests/minute, status classes and p50 latency parsed from Ollama's GIN access log (l to toggle, --logs)
• Sparklines — CPU, MEM, GPU utilization and VRAM history next to each bar, block characters, stdlib deque
• Scriptable — --json one-shot mode for cron or Ansible facts (exit code 1 on unhealthy); --json --watch streams NDJSON
• Prometheus exporter without a port — --prometheus prints text exposition format; --prometheus --watch -o …/textfile/mtop.prom keeps a node_exporter textfile fresh with atomic writes, no cron needed
• API-only mode — --no-docker for monitoring remote Ollama instances without local docker calls
• cgroup-aware CPU bar — normalizes against the container's --cpus/quota limit, not the host core count
• Docker and bare-metal — auto-detects the source: a Docker container, a systemd ollama.service, or a manual ollama serve; monitors process CPU/MEM via /proc (Linux, no root) or ps/sysctl (macOS)
• Docker Engine API over the socket — no docker binary needed: mtop talks to /var/run/docker.sock (or a rootless / Podman socket, or $DOCKER_HOST) with stdlib http.client; the docker/podman CLI is only a fallback. One-shot stats mean no 2 s docker stats stall per cycle
• Podman — same code path via the compat API socket or the podman CLI
• Respects $OLLAMA_HOST — works with remote Ollama instances out of the box
• Several instances at once — repeat -u; each remote gets its own LOADED MODELS table. Two Ollama instances on a mixed CUDA + ROCm host, or a fleet of remote boxes, on one screen
• Behind a reverse proxy — -H 'Authorization: Bearer …', $OLLAMA_API_KEY, basic auth in the URL, --insecure / --cacert for private certificates
• llama-swap + vLLM — --backend llama-swap: the whole model catalog with state, port and TTL, the vllm serve flags of each running model, the user-level systemd unit and its journal
• ComfyUI — --backend comfyui: the models of the running and recent jobs, live job progress (node, step x/y, ETA) from its websocket, the queue, the GPU memory ComfyUI holds, the /models catalog; --control cancels the job, clears the queue, frees the models
• Zero external dependencies — only Python stdlib (curses, urllib, json, subprocess)

Quick Start

One-liner (no install)

curl -fsSL https://github.com/Quaerendir/mtop/releases/latest/download/mtop.py -o mtop.py
chmod +x mtop.py
./mtop.py

pip install

pip install ollama-mtop
mtop

Or without touching the system Python: pipx install ollama-mtop / uv tool install ollama-mtop.

From source

git clone https://github.com/Quaerendir/mtop.git
cd mtop
pip install -e .
mtop

Run directly from a clone (no install)

git clone https://github.com/Quaerendir/mtop.git
cd mtop
PYTHONPATH=src python -m mtop

Usage

mtop [-c CONTAINER] [-i INTERVAL] [-u URL ...] [--backend B] [-H HEADER] [--insecure]
     [--cacert FILE] [-m MODE] [--runtime RT] [--no-gpu] [--control]
     [--json | --prometheus] [--watch] [-o FILE] [-V] [-h]

Options:
  -c, --container NAME   Docker container name (default: ollama, or llama-swap for
                         --backend llama-swap, comfyui for --backend comfyui)
  -m, --mode MODE        Data source: auto|docker|local|api (default: auto)
                         local = bare-metal `ollama serve` (systemd/proc/ps)
                         api   = models only, no host resource stats
  -i, --interval SECS    Refresh interval in seconds (default: 1.0)
  -u, --api-url URL      Ollama API base URL (default: $OLLAMA_HOST or http://localhost:11434)
                         Scheme-less values (gpu-rig:11434) are accepted, like Ollama itself.
                         Repeatable: the first is the primary (host stats, runners), the
                         rest are API-only. Optional label: -u rig=http://gpu-rig:11434.
                         Credentials in the URL (https://user:pw@host) become basic auth.
                         With --backend llama-swap the default is http://localhost:8001,
                         with --backend comfyui http://localhost:8188; $OLLAMA_HOST is
                         ignored for both.
      --backend B        Server to monitor: ollama|llama-swap|comfyui (default: ollama)
  -H, --header 'N: v'    Extra HTTP header for every API request (repeatable);
                         $OLLAMA_API_KEY is sent as 'Authorization: Bearer …' automatically
      --insecure         Skip TLS certificate verification for https:// endpoints
      --cacert FILE      CA bundle (PEM) to verify https:// endpoints against
      --runtime RT       How to reach the container runtime: auto|api|cli (default: auto)
                         api = Engine API on $DOCKER_HOST or a docker/podman socket
                         cli = docker/podman subprocesses
      --no-gpu           Disable GPU monitoring section
      --no-runners       Hide the RUNNERS section (effective inference config)
      --no-env           Hide the SERVER CONFIG section (OLLAMA_* environment)
      --logs             Show the LOGS section from the start (toggle with l); adds
                         `logs` with request stats to --json / --prometheus
      --log-lines N      Log lines to show (default: 8)
      --control          Allow model actions: arrow keys select a loaded model, s stops
                         (unloads) it after a y/N prompt, t keeps it loaded longer,
                         L loads one from /api/tags, P pulls one by name (X cancels).
                         Off by default — without it mtop never writes to the API
      --no-docker        API-only mode: skip all docker calls (remote instances)
      --json             Print one snapshot as JSON and exit (exit 1 on unhealthy)
      --prometheus       Print one snapshot in Prometheus text exposition format and exit
      --watch            With --json/--prometheus: emit every INTERVAL seconds until Ctrl-C
                         (--json --watch prints NDJSON, one compact object per line)
  -o, --output FILE      Write to FILE instead of stdout: Prometheus output replaces the file
                         atomically (tmp + rename), NDJSON is appended
  -V, --version          Show version
  -h, --help             Show help

Examples

# Monitor a custom container name
mtop -c my-ollama

# Slower refresh for remote/metered connections
mtop -i 5

# Bare-metal Ollama (systemd service or `ollama serve` in a terminal)
mtop --mode local

# Let mtop figure it out (docker? systemd? manual? — it probes in that order)
mtop

# Monitor a remote Ollama instance — API only, no local docker/GPU noise
mtop -u 192.168.1.100:11434 --mode api

# Several instances: the local one with full host stats, two remotes API-only
mtop -u localhost:11434 -u rig=gpu-rig:11434 -u spark=192.168.3.6:11434

# Ollama behind a reverse proxy with a private CA and a bearer token
OLLAMA_API_KEY=… mtop -u https://ollama.internal --cacert /etc/ssl/internal-ca.pem

# Podman (rootless): the socket is found automatically; or point at it
DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock mtop

# mtop inside a container, with only the socket mounted — no docker CLI needed
docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock \
  --network host python:3.12-slim sh -c \
  "curl -fsSL https://github.com/Quaerendir/mtop/releases/latest/download/mtop.py -o m.py && python m.py"

# One-shot health/state snapshot for scripting (schema_version=1 is the first key;
# it changes only when a field changes meaning or goes away)
mtop --json | jq '.models[].name'

# Stream one JSON line per second (NDJSON) — feed it to jq, a log shipper, or a file
mtop --json --watch -i 1 | jq -c '{t: .wallclock, models: [.models[].name]}'

# Prometheus, no port: keep a node_exporter textfile fresh (atomic writes)
mtop --prometheus --watch -i 15 -o /var/lib/node_exporter/textfile_collector/mtop.prom

# llama-swap + vLLM (user-level systemd unit, API on :8001)
mtop --backend llama-swap

# llama-swap on another port or host
mtop --backend llama-swap -u http://gpu-box:9292

# ComfyUI (container `comfyui` or a `python main.py` from a ComfyUI checkout, API on :8188)
mtop --backend comfyui

# Using OLLAMA_HOST environment variable
export OLLAMA_HOST=http://gpu-rig:11434
mtop

Interactive Keys

--control on the Spark, in real time (waits over 2 s cut): L loads two models from the picker (Ollama itself unloads the first when the second needs the memory), t 2 keeps one loaded for 2h, ? lists the keys, P pulls smollm:135m, s y stops a model. The recording is docs/demo-control.cast (asciinema play docs/demo-control.cast); tools/demo-control.sh records and renders it again.

What you see

The screenshot above is a real session on a DGX Spark (GB10) with Ollama in Docker and two models loaded. Top to bottom:

• header — host, container and the runtime in use (ollama · docker-api; or ollama: serve · pid, or the API URL in api mode), uptime, the Ollama version, and +N endpoints when several are polled; STALE appears if the collector falls behind
• CONTAINER / PROCESS RESOURCES — CPU normalized to the cgroup / systemd / affinity budget, memory with the accounting named (pss, rss or cgroup), sparklines of the recent history
• GPU — one block per card, any vendor; procs: lists what NVML sees on the card, joined to the runners by PID; ACPI is the hottest ACPI thermal zone, which on the GB10 reads differently from the GPU sensor
• LOADED MODELS — /api/ps, one table per endpoint when there are several; PROCESSOR computed exactly as ollama ps does
• RUNNERS — the negotiated inference config from each runner's argv, plus the card it sits on
• SERVER CONFIG — the OLLAMA_* environment the server was started with and where it was read from
• LOGS — tail of the container log or journal, with request rate and latency parsed from the GIN access log
• footer — keys in short form, toggles that are on in green; on a narrow terminal the view toggles go first, then the model keys, ? help always stays

o adds the raw ollama ps table. Refresh the screenshot with tools/screenshot.py (see its docstring).

llama-swap + vLLM

mtop --backend llama-swap monitors llama-swap, the proxy that starts and stops model servers on demand, here with vLLM behind it. The layout is the same; what feeds each section changes:

Defaults follow the backend: the API is http://localhost:8001 and the container name llama-swap, so on a box that runs both, --mode auto does not lock onto the unrelated ollama container. Pass -u / -c for anything else.

--control works here too: the cursor walks the whole catalog, s stops the selected model (POST /api/models/unload/<model>), L starts it (GET /upstream/<model>/health, which answers once the model is up; the start carries on if mtop exits). llama-swap runs one model at a time unless its config groups them, so L asks first while another model runs. There is no t or P: llama-swap takes TTL and its models from its config. Tested with llama-swap v256.

TTL counts down from the model's last request. llama-swap reports only the configured TTL, so mtop follows its /api/events stream (inflight requests, the /upstream/<model>/… access-log lines, models turning ready) and counts from what it sees. Until mtop has seen a model used, the column shows the configured TTL as an upper bound, e.g. ≤1d 0h. A one-shot --json / --prometheus has nothing to count from; with --watch they get ttl_left / expires_at and mtop_model_expires_seconds like Ollama.

Not available with this backend: the raw ollama ps table (o) and keep-loaded (t). Verified against llama-swap + vLLM and llama-server on an NVIDIA GB10.

ComfyUI

mtop --backend comfyui monitors ComfyUI, the node-based image and video generation server. Header, resources, GPU, SERVER CONFIG and LOGS work as for Ollama. The MODELS section is built differently, because ComfyUI is not a model server: it loads models as a job's workflow asks for them.

The screenshot is a real render of Qwen-Image on the Spark. Top to bottom in MODELS:

• queue — jobs running and waiting (/queue), and the GPU memory held by the ComfyUI process (NVML per-process memory; 0.3 GiB after /free, about 28 GiB with this workflow loaded)
• catalog — model files per folder from /models, re-read once a minute. Files under a Hugging Face hub cache (models--org--name/…), which ComfyUI lists in every folder an extra_model_paths entry points at, are counted once, apart
• job — the running job: node, sampler step x/y with a bar, elapsed time, ETA; when idle, the last job's outcome and duration (red when it failed or was interrupted)
• table — the models in the workflows of the running job and the last 20 finished ones

STATE. ComfyUI's API has no equivalent of /api/ps: nothing in it says which models are in memory. So the state says where mtop knows a model from:

Only in use models count as loaded for mtop_models_loaded.

Job progress. The node and its steps come from /ws, which mtop follows with a client id of its own. ComfyUI sends a job's progress to the client that queued it, and to everyone only when the job was queued without a client_id, as scripts posting to /prompt usually do. A job queued from the web UI belongs to the browser tab, so for it the line says running 1m 05s · progress goes only to the client that queued it. mtop never borrows the tab's id, as that would take the progress away from the tab. ComfyUI publishes no start time for a running job either, so the time counts from when mtop first found the job running. That is exact to one refresh, except for a job already running when mtop starts, which shows ≥. The ETA covers the current node's steps (the sampler: nearly the whole job) at the rate of the steps so far. One-shot --json / --prometheus read no websocket, so they have the time but no steps.

--control acts on the server, not on a model, and asks y/N first: s cancels the running job (POST /api/jobs/<id>/cancel, which interrupts only that job; an older ComfyUI gets POST /interrupt with the prompt id), C clears the pending queue (POST /queue {"clear": true}), F unloads every model and frees memory (POST /free). There is no cursor. With several -u, the actions go to the first endpoint.

Defaults follow the backend: the API is http://localhost:8188 and the container name comfyui. Pass -u / -c for anything else. Not available: the raw ollama ps table (o).

Prometheus adds mtop_comfyui_info, mtop_comfyui_queue_jobs{state}, mtop_comfyui_gpu_memory_bytes, mtop_comfyui_model_last_used_seconds{model,kind}, mtop_comfyui_job_elapsed_seconds{prompt_id}, mtop_comfyui_job_progress_ratio{prompt_id,node} and mtop_comfyui_websocket_up. Verified against ComfyUI 0.37.0 in Docker on an NVIDIA GB10: progress, cancel, clear and free on real renders.

Supported Platforms

Requirements

• Python 3.10+ (uses match-era type hints like list[str], X | Y)
• Docker or Podman for container monitoring — access to the socket is enough, the CLI is optional
• Ollama in a container, as a bare-metal process, or reachable via API — or llama-swap (--backend llama-swap), or ComfyUI (--backend comfyui)
• NVIDIA driver (optional, for GPU stats — libnvidia-ml.so.1, or nvidia-smi as fallback)

Roadmap

• Recorded --control session for the README (asciinema + agg)
• AMD GPU support (sysfs first, rocm-smi fallback)
• Apple Silicon GPU stats (via ioreg, no root — powermetrics needs sudo)
• Intel GPU stats via sysfs (partial: no utilization without CAP_PERFMON)
• Model pull with progress (P under --control)
• Multi-host support (repeatable -u, per-endpoint model tables)
• Docker Engine API over the socket, Podman support
• Configurable layout (raw ollama ps toggle; more sections to follow)
• Model actions — unload, load from /api/tags, extend TTL (--control)
• Sparkline history for CPU/GPU utilization (block chars, stdlib deque)
• systemd/bare-metal Ollama support (process stats via /proc, no Docker required)
• Log panel (container logs / journalctl) with request stats from the GIN log
• Effective inference config per runner (context, flash attention, KV dtype)
• Prometheus text exposition (--prometheus) and NDJSON streaming (--watch)
• Request rate from the access log (tokens/s is per-response data Ollama returns only to the caller — not observable from outside)
• Effective server environment (OLLAMA_*) per source
• Runner → GPU mapping via NVML per-process memory
• ComfyUI backend — models of the running and recent jobs, live job progress over /ws, cancel / clear / free (--backend comfyui)

Contributing

PRs welcome. Keep it stdlib-only — the zero-dependency constraint is a feature, not a limitation.

git clone https://github.com/Quaerendir/mtop.git
cd mtop
pip install -e ".[dev]"      # + pytest, ruff
# hack on src/mtop/*.py — util, procfs, runner, collector, ui, cli, container, gpu, logs, export, comfyui, comfywatch, wsclient
mtop

ruff check src tools tests
pytest                       # stdlib-only fakes: no GPU, docker or Ollama needed

# regenerate the single-file artifact shipped with releases
python tools/bundle.py       # -> dist/mtop.py

dist/mtop.py is generated — never edit it by hand. The version lives in src/mtop/_version.py only. A v* tag attaches the bundle to the GitHub release and, once the repository variable PYPI_TRUSTED is set, publishes the sdist and wheel to PyPI through Trusted Publishing (GitHub OIDC, no token in the repo). The tag must equal v + the package version.

License

MIT — see LICENSE.

Acknowledgements

Built as a collaboration between a human homelab geek and Claude (Anthropic) during a late-night infrastructure session. The original bash prototype migrated to Python/curses because fighting tput and jq in a loop was getting old.