onnx-perf-test 0.1.0

A simple tool to measure the performance of ONNX models in Python.

onnx-perf-test - ONNX Performance Test

A simple tool to measure the performance of ONNX models in Python.

Installation

pip install onnx-perf-test

Usage

python onnx_perf_test.py {onnx_model} [--provider {provider}] [--num_runs {num_runs}] [--output_dir {output_dir}] [--draw] [--keep_profiling_file]

Arguments:

• onnx_model: Path to the ONNX model file.
• --provider: Provider to use for inferencing. Default is uses onnxruntime.get_available_providers() to get the available providers. Options: TENSORRT, CUDA, CPU...
• --num_runs: Number of runs to average the performance. Default is 10.
• --output_dir: Output directory to save the results. Does not save the results if not provided.
• --draw: Draw the performance graph. Requires matplotlib to be installed. Default is False.
• --keep_profiling_file: Keep the profiling file generated by onnxruntime. Default is False.

Example

python onnx_perf_test.py model.onnx --provider CUDA --num_runs 20 --output_dir results --draw

Analyze ONNX Profiling File

Additionally, you can analyze your own .json profiling file generated by ONNXRuntime using the analyze_onnx_profiling.py script.

python analyze_onnx_profiling.py {onnx_profile_file} [--output_dir {output_dir}] [--draw]

Arguments:

• onnx_profile_file: Path to the ONNX profiling file.
• --output_dir: Output directory to save the results. Does not save the results if not provided.
• --draw: Draw the performance graph. Requires matplotlib to be installed. Default is False.

Example

python analyze_onnx_profiling.py model_profile.json --output_dir results --draw