cluster-smi

A terminal snapshot of Raspberry Pi cluster hardware health — the same idea as nvidia-smi, but for a rack (or desk) of Pis.

"How is my cluster's hardware doing right now?"

When you administer a Pi cluster you often need a fast answer to questions like:

• Which nodes are hot, throttling, or undervolted?
• Is one machine using far more CPU, memory, or network than its peers?
• Did a node reboot unexpectedly, fill its disk, or drop offline?
• Can I spot that without standing up Prometheus, Grafana, or agents?

cluster-smi answers those from a normal laptop or bastion host. It SSHs to each node, reads kernel interfaces (/proc, /sys), and prints a compact table you can glance at — or wrap in watch. No daemons, no database, no dashboards. Hardware-centric, not process-centric: there is no process viewer and no history.

Example

cluster-smi -h hosts.txt

HOST     CPU  TEMP  MEM  DISK  RX/s   TX/s   LOAD  THR  UPTIME
----------------------------------------------------------------
  pi01    21%  49°  31%  42%  3 MB   1 MB   0.41   -   4d
* pi02    98%  72°  81%  68% 41 MB   9 MB   5.02   T  11h
  pi03     3%  41°  18%  23%  0 MB   0 MB   0.04   -  18d
  pi04 OFFLINE

Rows marked with * look anomalous versus the rest of the cluster (or hit hard limits). THR flags Raspberry Pi throttling when the firmware reports it.

Drill into one node:

cluster-smi node pi02

Host            pi-node1
Model           Raspberry Pi 5 Model B Rev 1.1

Status          Online
Uptime          52m

CPU             1%
Frequency       2400 MHz
Temperature     53°C

Memory          302 MB / 16 GB
Disk            7%

Network RX      5 KB/s
Network TX      1 KB/s

Load            0.00
Throttle        NO
Undervoltage    NO

Quick start

Requires Python 3.10+ and OpenSSH (ssh).

1. List your nodes in ~/.config/cluster/hosts.txt (SSH Host aliases work):

mkdir -p ~/.config/cluster
cat > ~/.config/cluster/hosts.txt <<'EOF'
pi01
pi02
pi03
ubuntu@pi04
EOF

2. Run:

cluster-smi

Or refresh live:

watch -n 2 cluster-smi

Pass another file with -h hosts.txt when you need to. Demo without SSH: cluster-smi --mock.

SSH and macOS Keychain

Runs are non-interactive (BatchMode) — there is no passphrase prompt. Keys must already be available via the agent / Keychain.

On Apple systems, Keychain use is required. Prefer ~/.ssh/config:

Host node0
	HostName 192.168.129.36
	User admin
	IdentityFile ~/.ssh/id_ed25519_cluster
	AddKeysToAgent yes
	UseKeychain yes

Or once per session:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519_cluster_admin
ssh node0 true
cluster-smi

Without UseKeychain / --apple-use-keychain, you can get Permission denied (publickey,password) even when interactive ssh works.

Install

From this repository

User-level command (outside the project venv):

cd /path/to/cluster-smi
uv tool install .

Shim lands in ~/.local/bin — keep that on your PATH.

Editable (picks up local changes):

uv tool install --editable .

With pip:

pip install --user .

From PyPI

pip install cluster-smi

Options

How it works

Each poll runs one remote command per host over SSH. The collector emits a JSON snapshot of raw counters; CPU %, RX/s, and outlier detection are computed locally so peers stay comparable without any cluster-side agent.

Development

uv sync --extra dev
uv run pre-commit install
uv run pytest
uv run ruff check src tests
uv run ruff format src tests

Pre-commit runs the same checks as CI (Ruff lint/format + pytest) on each commit. Run everything against the whole tree with:

uv run pre-commit run --all-files