nvertake 0.1.0

A Python package for preemptive scheduling on NVIDIA GPUs

nVertake

A Python package for preemptive scheduling on NVIDIA GPUs.

Features

• Priority Scheduling: Elevate your GPU process priority using CPU nice values and CUDA stream priorities
• Memory Reservation: Reserve GPU memory to prevent other processes from claiming it
• Dynamic Memory Management: Maintain constant memory usage even during script execution

Installation

pip install -e .

Usage

Run with Elevated Priority

Run a Python script with elevated GPU scheduling priority:

nvertake run train.py --epochs 100

Memory Reservation

Reserve 95% of GPU memory while running your script:

nvertake --filled 0.95 run train.py

This prevents other processes from claiming GPU memory, even when your script temporarily frees memory during execution.

Standalone Memory Fill

Just fill GPU memory without running a script (useful for reserving GPU):

nvertake --filled 0.95

Press Ctrl+C to release the memory.

Specify GPU Device

nvertake --device 1 --filled 0.8 run inference.py

Show GPU Info

nvertake info

CLI Options

nvertake [OPTIONS] [COMMAND]

Options:
  --filled, -f RATIO    Fill GPU memory to this ratio (0.0-1.0)
  --device, -d GPU_ID   GPU device to use (default: 0)
  --nice, -n VALUE      Nice value for CPU priority (default: -10)
  --quiet, -q           Suppress info messages
  --version, -V         Show version

Commands:
  run SCRIPT [ARGS...]  Run a Python script with elevated priority
  info                  Show GPU information

How It Works

Priority Scheduling

1. CPU Priority: Uses os.nice() to increase process priority (lower nice value = higher priority)
2. CUDA Streams: Creates high-priority CUDA streams for GPU task scheduling

Memory Reservation

1. Calculates target memory based on fill ratio
2. Allocates buffer tensors to reach target
3. Runs a background thread to monitor and adjust buffer size
4. Maintains constant total GPU memory usage during script execution

Requirements

• Python >= 3.8
• PyTorch
• NVIDIA GPU with CUDA support
• psutil

License

MIT