nvProbe
NVIDIA GPU & CUDA Benchmark Suite
Automate CUDA workloads • HPL & HPCG • MLPerf inference • Custom kernels • Interactive reports
nvprobe.scszero.com
pip install nvprobe && nvprobe setup && nvprobe run --local
Features
| Bandwidth | MatMul / Attention | Conv2D |
|---|---|---|
| H2D / D2H / D2D across buffer sizes | fp32, fp16, int8 custom CUDA kernels | 2D convolution benchmarks |
| HPL (FP64 Linpack) | HPCG | MLPerf Inference |
| Datacenter GPUs: A100, H100, B200, L40S… | Conjugate Gradients | ONNX Runtime via cmx4mlperf |
- Bundled CUDA runtime — CuPy
[ctk]via pip, no system toolkit required - Auto-downloaded HPC tools — NVIDIA HPC Benchmarks cached in
~/.nvprobe/tools/ - Interactive HTML reports — Chart.js charts with GPU / transfer / precision dropdowns and oscilloscope-style glow
- A/B comparison — compare two result sets side-by-side
- Slurm integration — generate, submit, monitor, collect from HPC clusters
- SQLite storage — all results persisted; CSV / JSON export
Quick Start
| Step | Command | What it does |
|---|---|---|
| 1 | pip install nvprobe |
Install the package |
| 2 | nvprobe setup |
Install CuPy, download HPL/HPCG, generate configs |
| 3 | nvprobe env |
Verify GPU detection, driver, CUDA version |
| 4 | nvprobe run --local |
Run all benchmarks locally |
| 5 | nvprobe report --open |
Generate & open interactive HTML report |
Or from source:
git clone https://github.com/SergioZ3R0/nvprobe.git && cd nvprobe
pip install -e . && nvprobe setup && nvprobe run --local
More commands
| Command | Description |
|---|---|
nvprobe compare --a results/run1 --b results/run2 |
Compare two runs |
nvprobe run --config configs/cluster.yaml |
Run with custom YAML config |
nvprobe slurm submit --config configs/cluster.yaml |
Submit Slurm job |
nvprobe slurm status |
Check Slurm job status |
nvprobe setup --cuda 13 |
Setup with specific CUDA version |
Charts
Chart.js canvas-based charts with interactive controls:
- Bandwidth — filter by GPU and transfer type (H2D / D2H / D2D)
- MatMul / Attention — filter by GPU and precision (fp32 / fp16)
- Range slider — zoom into any x-axis region
- Moving average — smoother trend lines for dense data
YAML Config
name: my-run
gpu:
models: ["L40S", "B200"]
slurm:
enabled: true
partition: gpu
gpus_per_node: 8
precisions: [fp32, fp16]
benchmarks:
- name: bandwidth
params:
sizes_mb: [1, 4, 16, 64, 256, 1024]
- name: custom
params:
kernels: [matmul, attention]
Project Structure
nvprobe/
├── nvprobe/
│ ├── cli.py # CLI entry point
│ ├── config.py # YAML config loader
│ ├── runner.py # Benchmark orchestration
│ ├── slurm.py # Slurm job management
│ ├── reporter.py # Plotly HTML report generator
│ ├── db.py # SQLite storage + CSV/JSON export
│ └── benchmarks/
│ ├── base.py # Base class, GPU detection, diagnostics
│ ├── bandwidth.py # Memory bandwidth tests
│ ├── custom.py # Custom CUDA kernels
│ ├── hpl.py # HPL wrapper
│ ├── hpcg.py # HPCG wrapper
│ ├── mlperf.py # MLPerf via cmx4mlperf
│ └── _cuda/ # Raw CUDA kernels
├── configs/
│ ├── default.yaml
│ └── local.yaml
├── nvprobe.svg
├── index.html
├── README.md
└── pyproject.toml
Notes
- HPL / HPCG — NVIDIA HPC Benchmarks binaries are validated for datacenter GPUs (A100, H100, B200, L40S…). They may crash (SIGSEGV) on RTX series. Bandwidth and custom kernels work on any CUDA GPU.
- MLPerf cuDNN —
mlcrdiscovers cuDNN via system CUDA paths. If installed viapip install nvidia-cudnn-cuXX, pre-register with:mlcr get,cudnn,nvidia --input=$(python3 -c 'import nvidia.cudnn; print(nvidia.cudnn.__path__[0]'))
Requirements
Python 3.10+ • NVIDIA GPU with CUDA drivers • nvidia-smi in PATH • Slurm (optional)
License
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
nvprobe-0.7.1.tar.gz
(57.2 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
nvprobe-0.7.1-py3-none-any.whl
(66.7 kB
view details)
File details
Details for the file nvprobe-0.7.1.tar.gz.
File metadata
- Download URL: nvprobe-0.7.1.tar.gz
- Upload date:
- Size: 57.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.14.4
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6610f35ed49487e91ef7d813ca74d57c596781d677eee048561df39192ccc93a
|
|
| MD5 |
9c3c9cb270094215220c75645ffae601
|
|
| BLAKE2b-256 |
38a6181413d709e8a1bb411af4a67a500536f80265a0c040e191df30ea6302fa
|
File details
Details for the file nvprobe-0.7.1-py3-none-any.whl.
File metadata
- Download URL: nvprobe-0.7.1-py3-none-any.whl
- Upload date:
- Size: 66.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.14.4
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6eca6b464938734b2e272926ed6612a8477ac956193fa4d8e15e82247a64a355
|
|
| MD5 |
bc21db501d072bed1a2f30c3a5b8ec1f
|
|
| BLAKE2b-256 |
7faaf18f279c5ff2c16ca3155b1a2221dcc5f872f0e4fe0d82bf30a4a8f0edbc
|