xputop 0.1.6

An interactive, universal accelerator and system monitor for NPU, GPU, and TPU environments.

xputop

An interactive, universal accelerator and system monitor for terminal environments, supporting Huawei Ascend NPUs, NVIDIA GPUs, AMD GPUs, Intel XPUs, Google TPUs, and Custom backends. Inspired by nvitop and nvtop.

Features

• Universal Multi-Backend Support: Seamlessly monitors Huawei NPU, NVIDIA, AMD, Intel, and Google TPU. Supports configuring fallback or custom backends right from the configuration file using regex extractors !
• Rich Terminal UI: Beautiful per-device cards, memory usage bars, and active process lists.
• Fixed process row count per card to prevent UI jitter (-p N, default 3)
• Sparkline history curves (like nvtop) for Hardware, CPU, memory, disk, and network metrics
• Detail mode (-d) for combined chart + CPU + memory + disk view
• Full detail mode (-a) adds network I/O monitoring and system process tree
• Unified Hardware Telemetry: Automatically displays Power (W) and AICore/GPU Core Utilization (%) alongside VRAM constraints consistently across NPU, AMD, Intel, and NVIDIA.
• Background Async Logging: Run xputop -o logs.jsonl to silently record all metrics into JSONL/CSV datasets in the background without blocking or affecting training throughput!
• Offline HTML Visualization: Render your captured log datasets as a beautiful interactive HTML dashboard natively using xputop view logs.jsonl.
• Auto-Kill / Memory Safety Engine: Automatically SIGKILL top memory-consuming processes when VRAM or System RAM exceeds critical thresholds (Supports top N processes, dropping to an explicit limit, or targeted killing).
• System Telemetry: CPU per-core utilization, System memory (RAM + Swap), Disk usage paths, Network I/O.
• Threshold-based email alerting for both accelerator and system metrics.
• Configuration stored securely via xputop config.
• Lightweight design — zero extra subprocesses, strict timeout handles, instantaneous CPU sampling, and zero blocking I/O buffering.

Installation

# Minimal version (Hardware probing only)
pip install xputop

# Full feature-set (Includes system-monitoring tools & generic Database writers)
pip install "xputop[all]"

# Install with generic SQL database support only
pip install "xputop[db]"

Quick Start

# Launch the interactive TUI
xputop

# Detail mode: inline sparklines + CPU + memory + disk
xputop -d

# Enable gorgeous full 2D NVTop-style matrix line plotting (defaults to GPU and Memory)
xputop -P 

# Merge all GPUs and CPU onto a single unified 2D Plot overlay!
xputop -M -P cpu,gpu,mem

# Switch to grayscale or light themes
xputop -P --theme grayscale
xputop -M -P --theme light

# Record in CSV natively
xputop -d -o metrics.csv

# Record trace into SQLite database (Native, zero-dependency)
xputop -o metrics.sqlite --format sqlite

# Send logging data traces to remote MySQL DB! (Requires pip install 'xputop[db]')
xputop -o mysql+pymysql://user:pass@host/db

# Wait and let the format inference determine it natively based on suffix!
xputop -o logs.jsonl

# Automatically infer current date, saving to /logs/xputop_log_xxxxxxx.csv
xputop -o /logs/

# Control process display rows (default 3, 0=hide, -1=show all)
xputop -n 5 -P
xputop -n 0           # hide all processes

# Monitor specific disk paths
xputop -C -D /data /home

# Probe and list available backend states
xputop --list-backends
xputop --reset-backend

# Force specific backend (Will permanently cache setting to bypass auto-timeout logic on large clusters)
xputop --backend npu
xputop --backend nvidia

# Run in demo mode (no hardware needed)
xputop --demo -d

# Print a single snapshot and exit
xputop once --cpu --mem --disk

# Print snapshot as JSON
xputop once --json --cpu --mem --disk /data

# Show Chinese help
xputop --zh

# Generate a default configuration file
xputop config --generate

Command-Line Options

Short	Long	Default	Description
-V	--version	—	Show version and exit
-i	--interval	2.0	Refresh interval in seconds
	--config	—	Path to configuration file
-b	--backend	auto	Force hardware (npu, nvidia, amd, intel). Caches selection for faster boots!
	--list-backends	—	Print probed hardware tools
-C	--spark	off	Enable compact 1-line history sparklines (optional targets: cpu,gpu,mem)
-P	--plot	off	Enable full 2D matrix line plotting (optional targets: cpu,gpu,mem)
-M	--merge-plot	off	Superimpose 2D plots onto a single combined coordinate axis
	--theme	color	Set UI theme options (color, grayscale, dark, light)
-l	--chart-length	120	Number of history points for sparklines
-n	--procs	3	Process rows per card (0=hide, -1=all)
-c	--cpu	off	Show CPU per-core utilization panel
-m	--mem	off	Show system memory panel
-D	--disk	off	Show disk usage panel (optionally specify paths)
-o	--output	—	Setup logging output target (log.csv, metrics.sqlite, mysql...). Def: timestamped CSV in CWD
	--format	[auto]	Enforce layout format (jsonl, csv, sqlite, md, mysql...)
-d	--detail	off	Detail mode = --spark -c -m -D
-a	--detail-all	off	Full detail = -d + network + process tree
	--alert	—	Temporary alert overrides (e.g. mem_usage_percent=95.0)
	--kill	0.0	Auto-kill on memory alert. 0=disable(default), <0=Top
	--demo	off	Demo mode with fake AI hardware
	--zh	—	Show Chinese help

Custom Hardware Support

xputop supports writing flexible standard Regex patterns direct inside ~/.config/xputop/config.toml to extract monitoring variables from any CLI binary!

[backend.custom.my_chip]
probe_command = "custom-smi --version"
info_command = "custom-smi log --csv"
device_pattern = "Device (?P<id>\\d+) \\| (?P<name>\\S+) \\| Temp=(?P<temp>\\d+) \\| Power=(?P<power>[\\d.]+) \\| Util=(?P<util>\\d+)% \\| Mem=(?P<mem_used>\\d+)/(?P<mem_total>\\d+)"

Build & Release

See BUILD.md for development setup, building, and publishing instructions.

Requirements

• Python >= 3.8

License

Apache License 2.0