nvmon

A fancy NVIDIA GPU monitor for the terminal. One Python file, no dependencies.

Install

uv tool install nvmon    # or: pipx install nvmon

No uv? Install it, or use pip install nvmon inside whichever environment you use.

Server without internet: copy nvmon.py over and run it with Python 3.6+.

Start

nvmon              # Esc, q or Ctrl+C to quit
nvmon -i 1         # refresh every second (default: 0.5)
nvmon -g 0,2,4-7   # only these GPUs
ssh -t HOST nvmon  # over ssh: -t gives it a terminal

Reading a box

• Top edge: number and name, power against its limit, temperature, fan (if any), clock. Warnings only when something is off: SLOWED: power cap | too hot | hw brake, PCIe DEGRADED: x16 -> x8.
• Graph: utilization over time, newest on the right.
• GPU: how much of the time a kernel ran. cores / tensor (H100 and newer): how busy that kept the chip, and how much was Tensor Core math. One small kernel running nonstop reads GPU 100 % with cores at 30 %.
• MEM: memory in use. CPU -> GPU, GPU -> CPU: PCIe traffic against the link's top speed.
• Bottom edge: the processes, with run time and memory, by account and conda environment (yours unlabelled). + opens the process list.

The bottom line has the GPU numbers (click one, or press its key, to hide or show it), the keys, and a newer release when there is one.

Views: v

v goes through four views, V back. What does not fit scrolls (wheel, PgUp/PgDn).

Graphs: one under the other; two columns only when they would not fit, as here with eight GPUs in 31 lines.

Processes in the graph's place: account, run time, memory, command. The wheel scrolls them; Shift+wheel or ← → sideways.

Numbers only and GPU and MEM only: narrow, always one column, for a window beside another.

Numbers only, scrolled down	GPU and MEM only

Processes and jobs: p

A job is a process, or a launcher (torchrun, deepspeed…) with its workers.

• Pick one: click a process, or press Tab (Tab and the arrows move on). Its GPUs stand out, and the bottom line sums it up.
• f, or a double click: only its GPUs; again for all.
• p: every job in a list. Click a heading to sort, again to reverse (or s, r). ← → scroll the picked job's command. Esc, or a click on an empty spot, lets go.
• t stops a job of yours (SIGTERM), k kills it (SIGKILL): each after a y, and only if it is still the job you picked.

The list, busiest first, one job picked; and that job with f:

Themes: c

c goes through the themes, C back:

• nvmon, the default
• Hiroshige, Hokusai, O'Keeffe, Van Gogh, Bénédictus, Cassatt, Greek, Tam, Homer, Demuth: palettes of works at the Met, from MetBrewer
• Rosé Pine, from the editor theme
• black and white, and black on white

All keys

Hold Shift to select text.

It remembers

The view, theme, list order and hidden GPUs, for each computer you connect from, in ~/.config/nvmon/settings.json (%APPDATA%\nvmon on Windows). Delete it to start afresh.

Terminals

• Line characters drawn two cells wide (Xshell, PuTTY and the like, set up for Korean, Chinese or Japanese): nvmon notices and draws in ASCII. No 24-bit color reported: 256 colors.
• Boxes still broken: use a font with one-cell line characters (D2Coding, Cascadia Mono, JetBrains Mono, DejaVu Sans Mono) and UTF-8.
• Windows: use Windows Terminal. There is no mouse, and no t / k.
• nvmon asks PyPI once at start for a newer release; NVMON_NO_UPDATE_CHECK=1 turns that off.

License

MIT