Skip to main content

An utility to monitor NVIDIA GPU status and usage

Project description

gpustat

pypi Build Status license

Just less than nvidia-smi?

Screenshot: gpustat -cp

NOTE: This works with NVIDIA Graphics Devices only, no AMD support as of now. Contributions are welcome!

Self-Promotion: A web interface of gpustat is available (in alpha)! Check out gpustat-web.

Quick Installation

Install from PyPI:

pip install gpustat

If you don't have root (sudo) privilege, please try installing gpustat on user namespace: pip install --user gpustat.

To install the latest version (master branch) via pip:

pip install git+https://github.com/wookayin/gpustat.git@master

Please note that starting from v1.0, gpustat will support only Python 3.4+. For older versions (python 2.7, <3.4), you can continue using gpustat v0.x.

NVIDIA Driver Requirements

gpustat uses NVIDIA's official python bindings for NVML library (pynvml). As of now gpustat requires nvidia-ml-py >= 11.450.129, <=11.495.46, which is compatible with NVIDIA driver versions R450.00 or higher. Please upgrade the NVIDIA driver if gpustat fails to display process information. If your NVIDIA driver is too old, you can use older gpustat versions (pip install gpustat<1.0). See #107 for more details.

Usage

$ gpustat

Options (Please see gpustat --help for more details):

  • --color : Force colored output (even when stdout is not a tty)
  • --no-color : Suppress colored output
  • -u, --show-user : Display username of the process owner
  • -c, --show-cmd : Display the process name
  • -f, --show-full-cmd : Display full command and cpu stats of running process
  • -p, --show-pid : Display PID of the process
  • -F, --show-fan : Display GPU fan speed
  • -e, --show-codec : Display encoder and/or decoder utilization
  • -P, --show-power : Display GPU power usage and/or limit (draw or draw,limit)
  • -a, --show-all : Display all gpu properties above
  • --watch, -i, --interval : Run in watch mode (equivalent to watch gpustat) if given. Denotes interval between updates. (#41)
  • --json : JSON Output (Experimental, #10)

Tips

  • Try gpustat --debug if something goes wrong.
  • To periodically watch, try gpustat --watch or gpustat -i (#41).
    • For older versions, one may use watch --color -n1.0 gpustat --color.
  • Running nvidia-smi daemon (root privilege required) will make querying GPUs much faster and use less CPU (#54).
  • The GPU ID (index) shown by gpustat (and nvidia-smi) is PCI BUS ID, while CUDA uses a different ordering (assigns the fastest GPU with the lowest ID) by default. Therefore, in order to ensure CUDA and gpustat use same GPU index, configure the CUDA_DEVICE_ORDER environment variable to PCI_BUS_ID (before setting CUDA_VISIBLE_DEVICES for your CUDA program): export CUDA_DEVICE_ORDER=PCI_BUS_ID.

Default display

[0] GeForce GTX Titan X | 77°C,  96 % | 11848 / 12287 MB | python/52046(11821M)
  • [0]: GPU index (starts from 0) as PCI_BUS_ID
  • GeForce GTX Titan X: GPU name
  • 77°C: GPU Temperature (in Celsius)
  • 96 %: GPU Utilization
  • 11848 / 12287 MB: GPU Memory Usage (Used / Total)
  • python/...: Running processes on GPU, owner/cmdline/PID (and their GPU memory usage)

Changelog

See CHANGELOG.md

License

MIT License

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gpustat-1.0.0.tar.gz (90.5 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page