Repository of Intel® Low Precision Optimization Tool
Project description
Intel® Low Precision Optimization Tool
Intel® Low Precision Optimization Tool is an open-source python library which is intended to deliver a unified low-precision inference interface cross multiple Intel optimized DL frameworks on both CPU and GPU. It supports automatic accuracy-driven tuning strategies, along with additional objectives like performance, model size, or memory footprint. It also provides the easy extension capability for new backends, tuning strategies, metrics and objectives.
WARNING
GPU support is under development.
Currently supported Intel optimized DL frameworks are:
Currently supported tuning strategies are:
Introduction
Introduction explains Intel® Low Precision Optimization Tool infrastructure, design philosophy, supported functionality, details of tuning strategy implementations and tuning result on popular models.
Tutorials
- Hello World demonstrates the simple steps to utilize Intel® Low Precision Optimization Tool for quanitzation, which can help you quick start with the tool.
- Tutorials provides comprehensive instructions of how to utilize diffrennt features of Intel® Low Precision Optimization Tool.
- Examples is a tuning zoo to demonstrate the usage of Intel® Low Precision Optimization Tool in TensorFlow, PyTorch and MxNet for industry models of diffrent categories.
Install from source
git clone https://github.com/intel/lp-opt-tool.git
cd lp-opt-tool
python setup.py install
Install from binary
# install from pip
pip install ilit
# install from conda
conda config --add channels intel
conda install ilit
System Requirements
Hardware
Intel® Low Precision Optimization Tool supports systems based on Intel 64 architecture or compatible processors.
Software
Intel® Low Precision Optimization Tool requires to install Intel optimized framework version for TensorFlow, PyTorch, and MXNet.
Tuning Zoo
The followings are the examples integrated with Intel® Low Precision Optimization Tool for auto tuning.
TensorFlow Model | Category |
---|---|
ResNet50 V1 | Image Recognition |
ResNet50 V1.5 | Image Recognition |
ResNet101 | Image Recognition |
Inception V1 | Image Recognition |
Inception V2 | Image Recognition |
Inception V3 | Image Recognition |
Inception V4 | Image Recognition |
ResNetV2_50 | Image Recognition |
ResNetV2_101 | Image Recognition |
ResNetV2_152 | Image Recognition |
Inception ResNet V2 | Image Recognition |
SSD ResNet50 V1 | Object Detection |
Wide & Deep | Recommendation |
VGG16 | Image Recognition |
VGG19 | Image Recognition |
Style_transfer | Style Transfer |
PyTorch Model | Category |
---|---|
BERT-Large RTE | Language Translation |
BERT-Large QNLI | Language Translation |
BERT-Large CoLA | Language Translation |
BERT-Base SST-2 | Language Translation |
BERT-Base RTE | Language Translation |
BERT-Base STS-B | Language Translation |
BERT-Base CoLA | Language Translation |
BERT-Base MRPC | Language Translation |
DLRM | Recommendation |
BERT-Large MRPC | Language Translation |
ResNext101_32x8d | Image Recognition |
BERT-Large SQUAD | Language Translation |
ResNet50 V1.5 | Image Recognition |
ResNet18 | Image Recognition |
Inception V3 | Image Recognition |
YOLO V3 | Object Detection |
Peleenet | Image Recognition |
ResNest50 | Image Recognition |
SE_ResNext50_32x4d | Image Recognition |
ResNet50 V1.5 QAT | Image Recognition |
ResNet18 QAT | Image Recognition |
MxNet Model | Category |
---|---|
ResNet50 V1 | Image Recognition |
MobileNet V1 | Image Recognition |
MobileNet V2 | Image Recognition |
SSD-ResNet50 | Object Detection |
SqueezeNet V1 | Image Recognition |
ResNet18 | Image Recognition |
Inception V3 | Image Recognition |
Known Issues
-
KL Divergence Algorithm is very slow at TensorFlow
Due to TensorFlow not supporting tensor dump naturally, current solution of dumping the tensor content is adding print op and dumpping the value to stdout. So if the model to tune is a TensorFlow model, please restrict calibration.algorithm.activation and calibration.algorithm.weight in user yaml config file to minmax.
-
MSE tuning strategy doesn't work with PyTorch adaptor layer
MSE tuning strategy requires to compare FP32 tensor and INT8 tensor to decide which op has impact on final quantization accuracy. PyTorch adaptor layer doesn't implement this inspect tensor interface. So if the model to tune is a PyTorch model, please not choose MSE tuning strategy.
Support
Please submit your questions, feature requests, and bug reports on the GitHub issues page. You may also reach out to ilit.maintainers@intel.com.
Contributing
We welcome community contributions to Intel® Low Precision Optimization Tool. If you have an idea on how to improve the library:
- For changes impacting the public API, submit an RFC pull request.
- Ensure that the changes are consistent with the code contribution guidelines and coding style.
- Ensure that you can run all the examples with your patch.
- Submit a pull request.
For additional details, see contribution guidelines.
This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.
License
Intel® Low Precision Optimization Tool is licensed under Apache License Version 2.0. This software includes components with separate copyright notices and license terms. Your use of the source code for these components is subject to the terms and conditions of the following licenses.
Apache License Version 2.0:
MIT License:
See accompanying LICENSE file for full license text and copyright notices.
Citing
If you use Intel® Low Precision Optimization Tool in your research or wish to refer to the tuning results published in the Tuning Zoo, please use the following BibTeX entry.
@misc{Intel® Low Precision Optimization Tool,
author = {Feng Tian, Chuanqi Wang, Guoming Zhang, Penghui Cheng, Pengxin Yuan, Haihao Shen, and Jiong Gong},
title = {Intel® Low Precision Optimization Tool},
howpublished = {\url{https://github.com/intel/lp-opt-tool}},
year = {2020}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.