Skip to main content

Infrastructure for support neural networks compression

Project description

Model Compression Toolkit (MCT) Quantizers

The MCT Quantizers library is an open-source library developed by researchers and engineers working at Sony Semiconductor Israel.

It provides tools for easily representing a quantized neural network in both Keras and PyTorch. The library offers researchers, developers, and engineers a set of useful quantizers, along with a simple interface for implementing new custom quantizers.

High level description:

The library's quantizers interface consists of two main components:

  1. QuantizationWrapper: This object takes a layer with weights and a set of weight quantizers to infer a quantized layer.
  2. ActivationQuantizationHolder: An object that holds an activation quantizer to be used during inference.

Users can set the quantizers and all the quantization information for each layer by initializing the weights_quantizer and activation_quantizer API.

Please note that the quantization wrapper and the quantizers are framework-specific.

Quantizers:

The library provides the "Inferable Quantizer" interface for implementing new quantizers. This interface is based on the BaseInferableQuantizer class, which allows the definition of quantizers used for emulating inference-time quantization.

On top of BaseInferableQuantizer the library defines a set of framework-specific quantizers for both weights and activations:

  1. Keras Quantizers
  2. Pytorch Quantizers

The mark_quantizer Decorator

The @mark_quantizer decorator is used to assign each quantizer with static properties that define its task compatibility. Each quantizer class should be decorated with this decorator, which defines the following properties:

  • QuantizationTarget: An Enum that indicates whether the quantizer is intended for weights or activations quantization.
  • QuantizationMethod: A list of quantization methods (Uniform, Symmetric, etc.).
  • identifier: A unique identifier for the quantizer class. This is a helper property that allows the creation of advanced quantizers for specific tasks.

Getting Started

This section provides a quick guide to getting started. We begin with the installation process, either via source code or the pip server. Then, we provide a short example of usage.

Installation

Please refer to the MCT install guide for installing the pip package or building from the source.

From Source

git clone https://github.com/sony/mct_quantizers.git
python setup.py install

From PyPi - nightly package

Currently, only a nightly released package (unstable) is available via PyPi.

pip install mct-quantizers-nightly

Requirements

To use MCT Quantizers, you need to have one of the supported frameworks, Tensorflow or PyTorch, installed.

For use with Tensorflow, please install the following package: tensorflow,

For use with PyTorch, please install the following package: torch

You can also use the requirements file to set up your environment.

License

Apache License 2.0.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Built Distribution

File details

Details for the file mct-quantizers-nightly-1.2.0.20230831.post81152.tar.gz.

File metadata

File hashes

Hashes for mct-quantizers-nightly-1.2.0.20230831.post81152.tar.gz
Algorithm Hash digest
SHA256 d43f05f5b2eaa3423705b85a864f752c33af04721da81552910503ae936a8e44
MD5 7276f2bd7a54b60a862cca3ccb5ed364
BLAKE2b-256 f25b97aebb2d5ece5bbcfe1cf27021adfeedfff430ceae6be01a0693dd26fef7

See more details on using hashes here.

File details

Details for the file mct_quantizers_nightly-1.2.0.20230831.post81152-py3-none-any.whl.

File metadata

File hashes

Hashes for mct_quantizers_nightly-1.2.0.20230831.post81152-py3-none-any.whl
Algorithm Hash digest
SHA256 a71fde362e7561e69063af09ef439d9a5f28f259d5664aab8b607c243bc3e974
MD5 cfbc60111be74283bb9b360021795e83
BLAKE2b-256 9cb8a334d73becfd4f68ee72316e1b79be1a3e3851a7e8d8437cf992ab2e2a4c

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page