Model Compression Toolkit (MCT)
Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient hardware constrained.
This project enables researchers, developers and engineers an easy way to optimized and deploy state-of-the-art neural network on efficient hardware.
Specifically this project apply constrained quantization and pruning scheme on a neural network.
Currently, this project only support hardware friendly post training quantization (HPTQ) with Tensorflow 2 [1].
MCT project is developed by researchers and engineers working on the Sony Semiconductors Israel.
Table of Contents
Getting Started
This section provides a quick starting guide start with installtion via source code or pip server then a short example usage.
Installation
See the MCT install guide for the pip package, and build from source.
From Source
git clone https://github.com/sony/model_optimization.git
python setup.py install
From PyPi
pip install model-compression-toolkit
A nightly package is also available:
pip install mct-nightly
Example Usage
Here is a snapshot of a code that shown an example of how to use the post training quantization using keras.
import model_compression_toolkit as mct
# Set the batch size of the images at each calibration iteration.
batch_size = 50
# Create a representative data generator, which returns a list of images.
# Load a folder of images.
folder = '/path/to/images/folder'
# The images can be preprocessed using a list of preprocessing functions.
def normalization(x):
return (x - 127.5) / 127.5
# Create a FolderImageLoader instance which loads the images, preprocess them and enables you to sample batches of them.
image_data_loader = mct.FolderImageLoader(folder,
preprocessing=[normalization],
batch_size=batch_size)
# Create a Callable representative dataset for calibration purposes.
# The function should be called without any arguments, and should return a list numpy arrays (array for each model's input).
# For example: if the model has two input tensors - one with input shape of 32X32X3 and the second with input
# shape of 224X224X3, and we calibrate the model using batches of 20 images,
# calling representative_data_gen() should return a list
# of two numpy.ndarray objects where the arrays' shapes are [(20, 32, 32, 3), (20, 224, 224, 3)].
def representative_data_gen() -> list:
return [image_data_loader.sample()]
# Create a model and quantize it using the representative_data_gen as the calibration images.
# Set the number of calibration iterations to 10.
quantized_model, quantization_info = mct.keras_post_training_quantization(model,
representative_data_gen,
n_iter=10)
For more example please see the tutorials' directory.
For more information, please visit out project website.
Supported Features
Quantization:
* Post Training Quantization
* Gradient base post training using knowledge distillation (Experimental)
Tensorboard Visualization (Experimental):
* CS Analyizer: compare comprased model with orignal model to analysis large accuracy drop.
* Activation statisicis and errors
Note that currently we only have full support for Keras layers, using the TensorFlow native layer may lead to unexpected behavior. This limitation will be removed in future releases.
MCT is test with Tensorflow Version 2.5.
Tutorials and Results
As part of the MCT library, we have a set of example network on image classification which can be used as an example while using the package.
- Image Classification Example with MobileNet V1 on ImageNet dataset
| Network Name | Float Accuracy | 8Bit Accuracy | Comments |
|---|---|---|---|
| MobileNetV1 [2] | 70.558 | 70.418 |
For more results please see [1]
Contributions
MCT aims at keeping a more up-to-date fork and welcomes contributions from anyone.
*You will find more information about contributions in the Contribution guide.
License
Refernce
[1] Habi, H.V., Peretz, R., Cohen, E., Dikstein, L., Dror, O., Diamant, I., Jennings, R.H. and Netzer, A., 2021. HPTQ: Hardware-Friendly Post Training Quantization. arXiv preprint.
[2] MobilNet from Keras applications.
2 maintainers
Metadata
2 maintainers
Release files for mct-nightly 1.0.0.16112021-003049
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| mct-nightly-1.0.0.16112021-003049.tar.gz | 91.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| mct_nightly-1.0.0.16112021.post3049-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 268.3 kB
Release files / mct-nightly-1.0.0.16112021-003049.tar.gz
| Download URL | mct-nightly-1.0.0.16112021-003049.tar.gz |
|---|---|
| Size | 91.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8dc943c3f8f98ce2eac3f1f0eafd5da427c27d8924f4c0e2411b7e8b51f6f929
|
|
BLAKE2b-256 checksum How to use checksums |
f0dc1f46404787b9b2f41bfc2f7ec5dc75566ad09234411964834f6277b1ce04
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.12
|
Release files / mct_nightly-1.0.0.16112021.post3049-py3-none-any.whl
| Download URL | mct_nightly-1.0.0.16112021.post3049-py3-none-any.whl |
|---|---|
| Size | 176.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9d363007466f0d9cbbcbd0456696c4c2acb7443b3b2008edf9e1a6ddfac4e89d
|
|
BLAKE2b-256 checksum How to use checksums |
034721b1e085d69809997fc566599c32f35c1f4eeba8c52de65772d9ca80ba8e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.6.0 importlib_metadata/4.8.2 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.12
|
