Easy to use package for accelerate your pytorch model inference
Project description
TorchAccelerator
PyTorch model wrapper to accelerate models
How to use
from torch_accelerator import Model
# torch stuff
optimized_model = Model(model, input, trt_mode, pruning_coef)
# model: pytorch nn.module
# input: unput sample
# trt_mode: true if you want to use tensorrt engine inference (have to install torch2trt)
# pruning_coef: (from 0.1 to 1.0)
benchmark
2070super
==============================model: squeezenet1_0==============================
Model: squeezenet1_0_fp32
Using fp32 mode
squeezenet1_0_fp32 classification result: class 258 with probability 0.905274
fps: 601
Model: squeezenet1_0_fp16
Using fp16 mode
squeezenet1_0_fp16 classification result: class 258 with probability 0.903809
fps: 549
Model: squeezenet1_0_fp32_trt
Using fp32 mode
use tensorrt inference engine
squeezenet1_0_fp32_trt classification result: class 258 with probability 0.905484
fps: 1828
Model: squeezenet1_0_fp16_trt
Using fp16 mode
use tensorrt inference engine
squeezenet1_0_fp16_trt classification result: class 258 with probability 0.906738
fps: 1800
Model: squeezenet1_0_fp16_trt_prune10
Using fp16 mode
use tensorrt inference engine
pruning: 10.0 %
squeezenet1_0_fp16_trt_prune10 classification result: class 258 with probability 0.922363
fps: 1798
Model: squeezenet1_0_fp16_trt_prune30
Using fp16 mode
use tensorrt inference engine
pruning: 30.0 %
squeezenet1_0_fp16_trt_prune30 classification result: class 258 with probability 0.850586
fps: 1825
==============================model: squeezenet1_1==============================
Model: squeezenet1_1_fp32
Using fp32 mode
squeezenet1_1_fp32 classification result: class 258 with probability 0.930037
fps: 593
Model: squeezenet1_1_fp16
Using fp16 mode
squeezenet1_1_fp16 classification result: class 258 with probability 0.929199
fps: 558
Model: squeezenet1_1_fp32_trt
Using fp32 mode
use tensorrt inference engine
squeezenet1_1_fp32_trt classification result: class 258 with probability 0.930307
fps: 2973
Model: squeezenet1_1_fp16_trt
Using fp16 mode
use tensorrt inference engine
squeezenet1_1_fp16_trt classification result: class 258 with probability 0.930176
fps: 2625
Model: squeezenet1_1_fp16_trt_prune10
Using fp16 mode
use tensorrt inference engine
pruning: 10.0 %
squeezenet1_1_fp16_trt_prune10 classification result: class 258 with probability 0.938477
fps: 2754
Model: squeezenet1_1_fp16_trt_prune30
Using fp16 mode
use tensorrt inference engine
pruning: 30.0 %
squeezenet1_1_fp16_trt_prune30 classification result: class 258 with probability 0.833984
fps: 2635
==============================model: resnet18==============================
Model: resnet18_fp32
Using fp32 mode
resnet18_fp32 classification result: class 258 with probability 0.884896
fps: 595
Model: resnet18_fp16
Using fp16 mode
resnet18_fp16 classification result: class 258 with probability 0.884277
fps: 521
Model: resnet18_fp32_trt
Using fp32 mode
use tensorrt inference engine
resnet18_fp32_trt classification result: class 258 with probability 0.884760
fps: 1069
Model: resnet18_fp16_trt
Using fp16 mode
use tensorrt inference engine
resnet18_fp16_trt classification result: class 258 with probability 0.885254
fps: 1070
Model: resnet18_fp16_trt_prune10
Using fp16 mode
use tensorrt inference engine
pruning: 10.0 %
resnet18_fp16_trt_prune10 classification result: class 258 with probability 0.873535
fps: 1057
Model: resnet18_fp16_trt_prune30
Using fp16 mode
use tensorrt inference engine
pruning: 30.0 %
resnet18_fp16_trt_prune30 classification result: class 258 with probability 0.775879
fps: 1042
==============================model: resnet50==============================
Model: resnet50_fp32
Using fp32 mode
resnet50_fp32 classification result: class 258 with probability 0.873302
fps: 241
Model: resnet50_fp16
Using fp16 mode
resnet50_fp16 classification result: class 258 with probability 0.873047
fps: 210
Model: resnet50_fp32_trt
Using fp32 mode
use tensorrt inference engine
resnet50_fp32_trt classification result: class 258 with probability 0.873359
fps: 430
Model: resnet50_fp16_trt
Using fp16 mode
use tensorrt inference engine
resnet50_fp16_trt classification result: class 258 with probability 0.873047
fps: 426
Model: resnet50_fp16_trt_prune10
Using fp16 mode
use tensorrt inference engine
pruning: 10.0 %
resnet50_fp16_trt_prune10 classification result: class 258 with probability 0.875977
fps: 437
Model: resnet50_fp16_trt_prune30
Using fp16 mode
use tensorrt inference engine
pruning: 30.0 %
resnet50_fp16_trt_prune30 classification result: class 258 with probability 0.824219
fps: 409
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for torch_accelerator-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7159eb295da5918c80c537535f2c2e4ebb11081b4f26317683fba7bc839af861 |
|
MD5 | a51eec651bfb346cf914508a4a4a2482 |
|
BLAKE2b-256 | 26a634961bcebfa60cafc84f2e35b1c78a5e09aa8680e6864dc59482d6ef3e0b |