PySysMetrics

Real-time monitoring for CPU and GPU memory usage of Python processes.

PySysMetrics is a lightweight command-line tool and Python library that provides real-time monitoring of system resources consumed by Python processes. Track CPU memory, GPU memory (NVIDIA), CPU utilization, and thread count - perfect for profiling machine learning workflows, debugging memory leaks, and optimizing resource-intensive applications.

Features

• Real-time CPU Memory Monitoring - Track memory usage of all running Python processes
• GPU Memory Tracking - Monitor NVIDIA GPU memory consumption per Python process
• Detailed Process Information - View PID, script name, CPU percentage, thread count, and memory usage
• System Information Display - Shows OS details, CPU cores, total RAM, and GPU specifications
• Configurable Refresh Intervals - Set custom monitoring intervals (in seconds)
• Cross-Platform Support - Works on Linux, macOS (CPU only), and Windows
• Python API - Use as a library in your own scripts for programmatic monitoring
• Minimal Dependencies - Only requires psutil and pynvml (for GPU)

Use Cases

• Machine Learning Development - Monitor GPU memory during model training
• Performance Profiling - Track resource consumption of long-running scripts
• Memory Leak Detection - Identify memory growth patterns over time
• Multi-Process Applications - Monitor resource usage across parallel Python processes
• DevOps & Monitoring - Track Python service resource consumption
• Educational - Learn about system resource management and process monitoring

Installation

pip install pysysmetrics

Quick Start

Monitor CPU Memory Usage

# Monitor CPU memory every 2 seconds
pysysmetrics --cpu 2

Monitor GPU Memory Usage

# Monitor GPU memory every 5 seconds
pysysmetrics --gpu 5

CLI Documentation

Command Syntax

pysysmetrics [OPTIONS]

Options

Flag	Description	Example
--cpu N	Monitor CPU memory usage every N seconds	--cpu 2
--gpu N	Monitor GPU memory usage every N seconds	--gpu 5
-h, --help	Show help message and exit	-h

Note: If no arguments are provided, the help message will be displayed.

Usage Examples

# Monitor CPU memory usage with 2-second refresh
pysysmetrics --cpu 2

# Monitor GPU memory usage with 5-second refresh
pysysmetrics --gpu 5

# Display help
pysysmetrics --help

Output Examples

CPU Monitoring Output

When you run pysysmetrics --cpu 2, you'll see output like:

----------------------------------------------------------------------
System Information:
Operating System : Darwin 25.0.0
----------------------------------------------------------------------
CPU Information:
CPU Model        : arm
CPU Cores        : 10
Total RAM        : 16.00 GB
----------------------------------------------------------------------

========================= Refresh @ 2025-01-15 14:23:45 =========================
PID       Script Name              CPU %     #Threads  Memory (MB)
--------------------------------------------------------------------------------
12345     train_model.py           85.2      8         2048.50
12346     data_processor.py        12.5      4         512.25
12347     api_server.py            5.3       2         256.10
======================================================================

Output Fields:

• PID - Process ID
• Script Name - Name of the Python script (extracted from command line)
• CPU % - Current CPU utilization percentage
• #Threads - Number of threads used by the process
• Memory (MB) - Resident Set Size (RSS) memory in megabytes

GPU Monitoring Output

When you run pysysmetrics --gpu 5, you'll see output like:

----------------------------------------------------------------------
System Information:
Operating System : Linux 5.15.0
----------------------------------------------------------------------
GPU Information:
GPU 0 Model        : NVIDIA GeForce RTX 3090
GPU 0 Total Memory : 24.00 GB (24576 MB)
----------------------------------------------------------------------

========================= Refresh @ 2025-01-15 14:25:30 =========================
PID       Script Name                   GPU Mem (MB)   GPU Mem (GB)   GPU
--------------------------------------------------------------------------------
23456     train_model.py                8192.00        8.00           GPU 0
23457     inference.py                  2048.00        2.00           GPU 0
======================================================================

Output Fields:

• PID - Process ID
• Script Name - Name of the Python script
• GPU Mem (MB) - GPU memory used in megabytes
• GPU Mem (GB) - GPU memory used in gigabytes
• GPU - GPU device index

Python API

You can use PySysMetrics programmatically in your Python scripts:

CPU Monitoring

from pysysmetrics.core import cpu

# Monitor CPU memory usage every 3 seconds
cpu.monitor_cpu_memory(interval=3)

This will continuously monitor and print CPU memory statistics every 3 seconds.

GPU Monitoring

from pysysmetrics.core import gpu

# Monitor GPU memory usage every 2 seconds
gpu.monitor_gpu_memory(interval=2)

This will continuously monitor and print GPU memory statistics every 2 seconds.

Advanced Usage - Get Process Data

from pysysmetrics.core.cpu import get_python_processes, print_python_processes_info

# Get list of all Python processes
processes = get_python_processes()

# Print formatted process information
print_python_processes_info(processes)

from pysysmetrics.core.gpu import get_gpu_processes, print_gpu_processes_info

# Get list of Python processes using GPU
gpu_processes = get_gpu_processes()

# Print formatted GPU process information
print_gpu_processes_info(gpu_processes)

Requirements

• Python: 3.7 or higher
• CPU Monitoring: Works on all platforms (Linux, macOS, Windows)
• GPU Monitoring:
  • NVIDIA GPU with CUDA support
  • NVIDIA drivers installed
  • Working nvidia-smi command
  • Not supported on macOS (pynvml limitation)

Dependencies

PySysMetrics has minimal dependencies:

• psutil - Cross-platform process and system monitoring
• pynvml - Python bindings for NVIDIA Management Library (GPU monitoring)

These are automatically installed when you pip install pysysmetrics.

Project Structure

pysysmetrics/
├── pysysmetrics/
│   ├── __init__.py
│   ├── cli/
│   │   ├── __init__.py
│   │   └── main.py              # CLI entry point and argument parsing
│   ├── core/
│   │   ├── __init__.py
│   │   ├── cpu.py               # CPU memory monitoring functions
│   │   └── gpu.py               # GPU memory monitoring functions
│   └── utils/
│       ├── __init__.py
│       ├── test_cpu.py          # CPU stress testing utility
│       └── test_gpu.py          # GPU stress testing utility
├── setup.py                     # Package configuration
├── pyproject.toml               # Build system configuration
├── README.md                    # This file
└── push.sh                      # Build and publish script

Testing Utilities

PySysMetrics includes stress testing utilities to generate CPU and GPU load for testing the monitoring tools.

CPU Stress Test

python -m pysysmetrics.utils.test_cpu

This script creates CPU and RAM load cycles:

• 5 seconds of CPU stress (busy-wait loop)
• 5 seconds of RAM allocation (~500MB)
• 5 seconds idle
• Repeats indefinitely

GPU Stress Test

python -m pysysmetrics.utils.test_gpu

This script creates GPU load cycles using PyTorch:

• 5 seconds of GPU stress (matrix multiplication on CUDA)
• 5 seconds idle
• Repeats indefinitely

Note: GPU stress test requires PyTorch with CUDA support installed separately:

pip install torch

Testing Workflow

1. Terminal 1 - Run stress test:

   python -m pysysmetrics.utils.test_cpu
   

2. Terminal 2 - Monitor the process:

   pysysmetrics --cpu 1
   

You should see the test script appear in the monitoring output with resource usage.

Platform-Specific Notes

Linux

• Full support for both CPU and GPU monitoring
• GPU monitoring requires NVIDIA drivers and CUDA

macOS

• CPU monitoring fully supported
• GPU monitoring not supported (pynvml limitation)
• Uses system psutil library for process detection

Windows

• CPU monitoring fully supported
• GPU monitoring requires NVIDIA drivers
• May need to run with administrator privileges for full process access

Known Limitations

• GPU Monitoring on macOS - Not supported due to pynvml library limitations
• Script Name Detection - May show "N/A" for processes without .py files in command line
• Permission Issues - Some processes may not be accessible without elevated privileges
• NVIDIA Only - GPU monitoring only works with NVIDIA GPUs (uses NVML)
• Monitoring Overhead - Very minimal, but does consume some CPU for process enumeration

Troubleshooting

Issue: "No module named 'pynvml'"

Solution: Install pynvml: pip install pynvml

Issue: "NVIDIA driver or GPU not detected"

Solution:

• Verify NVIDIA drivers are installed: nvidia-smi
• Ensure you have an NVIDIA GPU
• Check that CUDA is properly installed

Issue: "GPU monitoring is not supported on macOS"

Solution: Use CPU monitoring instead (--cpu). GPU monitoring requires Linux or Windows with NVIDIA GPU.

Issue: "No active Python processes found"

Solution:

• Run a Python script in another terminal
• Use the included test utilities to generate load
• Ensure processes are running and accessible

Issue: "Permission denied" errors

Solution:

• Run with sudo (Linux/macOS): sudo pysysmetrics --cpu 2
• Run as Administrator (Windows)
• Note: Some system processes may still be inaccessible

Contributing

Contributions are welcome! Here's how you can help:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes
4. Test thoroughly - Ensure CPU and GPU monitoring work correctly
5. Commit your changes (git commit -m 'Add amazing feature')
6. Push to the branch (git push origin feature/amazing-feature)
7. Open a Pull Request

Development Setup

# Clone the repository
git clone https://github.com/yourusername/pysysmetrics.git
cd pysysmetrics

# Install in development mode
pip install -e .

# Test the CLI
pysysmetrics --cpu 2

License

This project is licensed under the MIT License - see the LICENSE file for details.

Changelog

Version 0.1.4

• Comprehensive README with complete documentation
• Enhanced package structure
• Improved testing utilities

Version 0.1.2

• CPU core count and GPU percentage improvements
• Enhanced system information display

Version 0.1.0

• Initial release
• CPU memory monitoring
• GPU memory monitoring
• CLI interface

Acknowledgments

• Built with psutil for cross-platform process monitoring
• Uses pynvml for NVIDIA GPU management
• Inspired by tools like nvidia-smi, htop, and top
• Built for the Python and ML community

Roadmap

• Support for AMD GPUs (ROCm)
• Historical data logging and visualization
• Export metrics to CSV/JSON
• Web dashboard for remote monitoring
• Docker container support
• Process filtering and search
• Alert thresholds for high memory usage
• Integration with Prometheus/Grafana

Support

• Author: Aditya Thiyyagura
• Email: thiyyaguraadityareddy@gmail.com
• Issues: Report bugs and request features on GitHub

---

Made with Python by Aditya Thiyyagura