Model Compression Tollkit (MCT) Quantizers
This is an open-source library that provides tools that enable to easily represent a quantized neural network, both in Keras and in PyTorch. It provides researchers, developers, and engineers a set of useful quantizers and, in addition, a simple interface for implementing new custom quantizers.
The MCT Quantizers library is developed by researchers and engineers working at Sony Semiconductor Israel.
High level description
The quantizers interface is composed of two main components:
QuantizationWrapper- an object that takes a layer with weights and a set of weights quantizers and infer a quantized layer.ActivationQuantizationHolder- an object that holds an activation quantizer to be quantized during inference.
The quantizers and all the quantization information for each layer can be set by initializing the weights_quantizer and activation_quantizer API.
Notice that the quantization wrapper and the quantizers are per framework.
Quantizers
The library defines the "Inferable Quantizer" interface for implementing new quantizers.
It is based on the basic class BaseInferableQuantizer which allows to define quantizers that are used for emulating inference-time quantization.
On top of BaseInferableQuantizer we define a set of framework-specific quantizers for both weights and activations:
The mark_quantizer Decorator
The @mark_quantizer decorator is used to supply each quantizer with static properties which define its task compatibility. Each quantizer class should be decorated with this decorator. It defines the following properties:
QuantizationTarget: An Enum that indicates whether the quantizer is designated for weights or activations quantization.QuantizationMethod: A list of quantization methods (Uniform, Symmetric, etc.).quantizer_type: An optional property that defines the type of the quantization technique. This is a helper property to allow creating advanced quantizers for specific tasks.
Getting Started
This section provides a quick starting guide. We begin with installation via source code or pip server. Then, we provide a short usage example.
Installation
See the MCT install guide for the pip package, and build from the source.
From Source
git clone https://github.com/sony/mct_quantizers.git
python setup.py install
From PyPi - nightly package
Currently, only a nightly released package (unstable) is available via PyPi.
pip install mct-quantizers-nightly
Requirements
To use MCT Quantizers, one of the supported frameworks, Tensorflow/PyTorch, needs to be installed.
For use with Tensorflow please install the packages: tensorflow, tensorflow-model-optimization
For use with PyTorch please install the packages: torch
Also, a requirements file can be used to set up your environment.
License
2 maintainers
Metadata
2 maintainers
Release files for mct-quantizers-nightly 1.0.0.420230608.post123559
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| mct-quantizers-nightly-1.0.0.420230608.post123559.tar.gz | 32.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| mct_quantizers_nightly-1.0.0.420230608.post123559-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 109.9 kB
Release files / mct-quantizers-nightly-1.0.0.420230608.post123559.tar.gz
| Download URL | mct-quantizers-nightly-1.0.0.420230608.post123559.tar.gz |
|---|---|
| Size | 32.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4d8bc5534e84bd189f615c7100dc3c3ea0f9431506a061c5db9d83a42f27aa49
|
|
BLAKE2b-256 checksum How to use checksums |
fad6b03d6bc87f7f5ce87da754d399208e4b2b7e207b51c7d598c3aaa2cd992e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.8.16
|
Release files / mct_quantizers_nightly-1.0.0.420230608.post123559-py3-none-any.whl
| Download URL | mct_quantizers_nightly-1.0.0.420230608.post123559-py3-none-any.whl |
|---|---|
| Size | 77.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
399a290e45fe5bc10329783dbda90bd69a1073cc5746d5bd946626b62fe3a6cc
|
|
BLAKE2b-256 checksum How to use checksums |
b55d10d1c7fd145467803bc97a45002915132592fe24f69ca65337c43b3ae0c1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.8.16
|
