trtutils
A high-level Python interface for TensorRT inference, providing a simple and unified way to run arbitrary TensorRT engines. This library abstracts away the complexity of CUDA memory management, binding management, and engine execution, making it easy to perform inference with any TensorRT engine.
Features
- Simple, high-level interface for TensorRT inference
- Automatic CUDA memory management
- Support for arbitrary TensorRT engines
- Built-in preprocessing and postprocessing capabilities
- Comprehensive type hints and documentation
- Support for both basic engine execution and end-to-end model inference
Quick Start
Basic Engine Usage
The TRTEngine class provides a simple interface for running any TensorRT engine:
from trtutils import TRTEngine
# Load your TensorRT engine
engine = TRTEngine("path_to_engine")
# Get input specifications
print(engine.input_shapes) # Expected input shapes
print(engine.input_dtypes) # Expected input data types
# Run inference
inputs = read_your_data()
outputs = engine.execute(inputs)
Installation
pip install trtutils
For additional features, you can install optional dependencies:
# For JIT compiler
pip install "trtutils[jit]"
# For development
pip install "trtutils[dev]"
Documentation
For detailed documentation, including advanced usage, examples, and API reference, visit our documentation site.
Examples
Check out our examples directory for more detailed usage examples, including:
- Basic engine usage
- End-to-end model inference
- YOLO model implementation
- Benchmarking utilities
Performance
| Device | YOLOv8m | YOLOv8n |
|---|---|---|
| OrinAGX-64GB |  |
 |
| OrinAGX-32GB |  |
 |
| OrinNX-16GB |  |
 |
| OrinNano-8GB |  |
 |
| XavierNX-8GB |  |
 |
| 3080Ti |  |
 |
| TitanRTX |  |
 |
License
This project is licensed under the MIT License - see the LICENSE file for details.
1 maintainer
Metadata
1 maintainer
Release files for trtutils 0.6.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| trtutils-0.6.1.tar.gz | 80.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| trtutils-0.6.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 186.1 kB
Release files / trtutils-0.6.1.tar.gz
| Download URL | trtutils-0.6.1.tar.gz |
|---|---|
| Size | 80.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
31092c436f354874289f9f90b627e9c7e07aa787bf5f72630b8eb732f96caa66
|
|
BLAKE2b-256 checksum How to use checksums |
1fb51fb8fa63f204ccd872c8b69b3593932c30a537e44034fe31d86147a8998a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.9
|
Release files / trtutils-0.6.1-py3-none-any.whl
| Download URL | trtutils-0.6.1-py3-none-any.whl |
|---|---|
| Size | 106.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
e41f5c7a5c59d3140d5f0ec56b6682e3fea6dbcdbac84d69ef27b32cb0f9cd1a
|
|
BLAKE2b-256 checksum How to use checksums |
2436f6d5d4ff6a44957f8698d15141f8c9a7421851387e4771da7f8d4359b9f5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.9
|
