smon

A terminal user interface (TUI) for monitoring Slurm clusters. Built with Textual for DGX H100 clusters.

uvx --from smon-tui smon

Features

• Job monitoring with live updates
• Node status display with GPU availability
• GPU count display with partition info
• Script viewer with syntax highlighting
• Output tracking (stdout/stderr)
• Search and filtering
• Tabbed TUI interface
• Keyboard shortcuts
• gpustat-web integration for real-time GPU monitoring

Installation

smon is on PyPI as smon-tui (the name smon was taken). The installed command is smon.

Requires Python 3.11+ on a machine where squeue, sinfo, scontrol and scancel are on PATH (typically a Slurm login node). Try it without a cluster with smon --mock.

Usage

Basic Usage

smon

Command Line Options

smon --help                    # Show help
smon --refresh 10              # Set refresh interval to 10 seconds
smon --user alice              # Filter jobs by user
smon --me                      # Filter jobs by the current user
smon --partition gpu           # Filter jobs and nodes by partition
smon --state RUNNING           # Default job state filter
smon --theme light             # Colour theme (dark/light)
smon --gpustat-web URL         # Enable gpustat-web integration
smon --mock                    # Demo mode without a Slurm cluster

Every option can also be set in ~/.config/smon/config.json (refresh_sec, user_filter, partition_filter, state_filter, theme, gpustat_web_url); command line flags take precedence.

Keyboard Shortcuts

TUI Interface

Jobs Tab

• Left: job table (JobID, user, state, partition, CPUs, memory, GPUs, time used vs. limit, nodes)
• Right: scontrol show job details, the batch script with syntax highlighting, and the tail of stdout/stderr
• The table updates in place on each refresh, so scrolling, sorting and the selected row are preserved

Nodes Tab

• Node status, GPU allocation bar, CPU and memory usage per node
• Enter on a node lists the jobs running on it
• gpustat-web integration (side-by-side view)

gpustat-web Integration

smon can display real-time GPU status from gpustat-web alongside the Slurm node information.

Setup

1. Make sure gpustat-web is running on your cluster (e.g., http://10.50.0.111:48109/)

2. Run smon with the --gpustat-web option:

   smon --gpustat-web http://10.50.0.111:48109/
   

3. Or add to config file (~/.config/smon/config.json):

   {
     "gpustat_web_url": "http://10.50.0.111:48109/"
   }
   

The Nodes tab will show the Slurm node table on the left and live GPU status from gpustat-web on the right.

Troubleshooting

Clicks land in the top-left corner / open the command palette. Some terminal multiplexers (herdr ≤ 0.9.0, herdrdev/herdr#3295) advertise pixel-coordinate mouse reporting but keep sending cell coordinates, which makes Textual apps map every click to the first cell. smon therefore does not negotiate pixel mouse mode at all; set SMON_PIXEL_MOUSE=1 to restore Textual's default if you need sub-cell precision. Upgrading the multiplexer (herdr ≥ 0.9.1) fixes it upstream as well.

Built with

• Textual - TUI framework
• Rich - Text formatting

Related projects

• https://github.com/wjwei-handsome/Slurmer

Contributing

Issues and pull requests are welcome.

git clone https://github.com/MilkClouds/smon.git && cd smon
uv sync --dev
uv run smon --mock      # run against mock data
uv run pytest           # tests
uv run ruff check .     # lint

Releasing

Versions come from git tags (hatch-vcs). Pushing a v* tag builds and publishes to PyPI via trusted publishing:

git tag -a v0.2.0 -m "v0.2.0" && git push origin v0.2.0