mct-quantizers-nightly 1.0.0.8062023.post1051

Infrastructure for support neural networks compression

Model Compression Tollkit (MCT) Quantizers

This is an open-source library that provides tools that enable to easily represent a quantized neural network, both in Keras and in PyTorch. It provides researchers, developers, and engineers a set of useful quantizers and, in addition, a simple interface for implementing new custom quantizers.

The MCT Quantizers library is developed by researchers and engineers working at Sony Semiconductor Israel.

High level description

The quantizers interface is composed of two main components:

1. QuantizationWrapper - an object that takes a layer with weights and a set of weights quantizers and infer a quantized layer.
2. ActivationQuantizationHolder - an object that holds an activation quantizer to be quantized during inference.

The quantizers and all the quantization information for each layer can be set by initializing the weights_quantizer and activation_quantizer API.

Notice that the quantization wrapper and the quantizers are per framework.

Quantizers

The library defines the "Inferable Quantizer" interface for implementing new quantizers. It is based on the basic class BaseInferableQuantizer which allows to define quantizers that are used for emulating inference-time quantization.

On top of BaseInferableQuantizer we define a set of framework-specific quantizers for both weights and activations:

1. Keras Quantizers
2. Pytorch Quantizers

The mark_quantizer Decorator

The @mark_quantizer decorator is used to supply each quantizer with static properties which define its task compatibility. Each quantizer class should be decorated with this decorator. It defines the following properties:

• QuantizationTarget: An Enum that indicates whether the quantizer is designated for weights or activations quantization.
• QuantizationMethod: A list of quantization methods (Uniform, Symmetric, etc.).
• quantizer_type: An optional property that defines the type of the quantization technique. This is a helper property to allow creating advanced quantizers for specific tasks.

Getting Started

This section provides a quick starting guide. We begin with installation via source code or pip server. Then, we provide a short usage example.

Installation

See the MCT install guide for the pip package, and build from the source.

From Source

git clone https://github.com/sony/mct_quantizers.git
python setup.py install

From PyPi - nightly package

Currently, only a nightly released package (unstable) is available via PyPi.

pip install mct-quantizers-nightly

Requirements

To use MCT Quantizers, one of the supported frameworks, Tensorflow/PyTorch, needs to be installed.

For use with Tensorflow please install the packages: tensorflow, tensorflow-model-optimization

For use with PyTorch please install the packages: torch

Also, a requirements file can be used to set up your environment.

License

Apache License 2.0.