nvbenjo 0.0.1

Nvbenjo is a utility for benchmarking inference of deep learning models on NVIDIA GPUs.

Nvbenjo

Nvbenjo is a utility for benchmarking inference of deep learning models on NVIDIA GPUs. It supports models in Onnx format as well as PyTorch models.

Usage

# Specify models to run in the command line
nvbenjo \
"+nvbenjo.models={\
    efficientnet: {type_or_path: 'torchvision:efficientnet_b0',  shape:['B',3,224,224],  batch_sizes: [16,32]},\
    resnet:       {type_or_path: 'torchvision:wide_resnet101_2', shape: ['B',3,224,224], batch_sizes: [16,32]}\
}"

# or better, specify your own config (or one of the pre-defined config files)
nvbenjo -cn small
nvbenjo -cn="/my/config/path/myconfig.yaml"

# override single arguments of your config
nvbenjo -cn="/my/config/path/myconfig.yaml" nvbenjo.models.mymodel.num_batches=10

# show current config and help
nvbenjo -cn="/my/config/path/myconfig.yaml" --help

Development

Example using uv:

uv sync --extra dev --extra onnx-cpu # or gpu
uv run nvbenjo

# for a quick run
uv run nvbenjo -cn small

# tests
uv run pytest
uv run nox