Aidge Export: TensorRT

The aim of this module is to provide an export to TensorRT SDK via the Aidge framework.

Quick start

System Requirements

• python >= 3.10
• aidge_core

pip install aidge-export-cpp

To compile the export, you need one of the following:

• Docker installed (recommended). The Docker image includes TensorRT, so no additional installation is required.
• TensorRT 8.6 or 10.10 installed on your host machine (required only if compiling directly on the host without Docker).

🛠 Build from Source

Prerequisite (in addition to previous one):

• Please review the global installation instructions before proceeding.
• If using a virtual environment, use the same one for all Aidge modules.

1. Python installation using setup scripts

[!TIP] Use Get-Help setup.ps1 (Windows) or ./setup.sh -h (Unix) for full documentation.

2. Python Installation using pip

Run these commands from the aidge_export_cpp/ directory:

#fStandard install
pip install . -v

# Install with testing dependencies
pip install .[test] -v && pytest

Editable Install (Experimental)

Use this for real-time development without re-installing.

pip install --no-build-isolation -ve . --config-settings=editable.rebuild=true -Cbuild-dir=build

Usage

aidge_export_tensorrt module exposes its main python API using export() function.

import aidge_export_tensorrt
aidge_export_tensorrt.export("export_trt", "model.onnx")

The export provides a Makefile with several options to utilize the export on your machine. You can generate either a C++ export or a Python export.
Additionally, you have the option to compile the export and/or the Python library using Docker if your host machine lacks the necessary packages.

The available commands are summarized in the following table:

Here's an example to compile and test the export Python library using Docker:

cd export_trt/ 
make build_lib_python_docker
make test_lib_python_docker

This will execute the test.py file within the Docker container, initializing and profiling the selected model.

Known issue

Export side

Issue related to the usage of the TensorRT export.

No CMAKE_CUDA_COMPILER could be found

CMake Error at CMakeLists.txt:21 (enable_language):
  No CMAKE_CUDA_COMPILER could be found.

  Tell CMake where to find the compiler by setting either the environment
  variable "CUDACXX" or the CMake cache entry CMAKE_CUDA_COMPILER to the full
  path to the compiler, or to the compiler name if it is in the PATH.

This error occur when you try to compile your project without having NVCC to your PATH.

To fix this, add nvcc to the path:

export PATH=<NVCC_PATH>:$PATH;

Where <NVCC_PATH> is the path to the nvcc compiler. For recent ORIN nvcc is installed at: /usr/local/cuda/bin.