Skip to main content

Distributional Quantization

Project description

Distributional Quantization

distquant_logo

Distributional quantization provides an information theoretically optimal and efficient quantization of data following a distributional prior.

This repository includes a pre-print of an excerpt of the distributional quantization paper.

Simulation plots can be found here.

The distquant package requires PyTorch and can be installed via

pip install distquant

💻 Usage

The distquant package can be used to efficiently and effectively quantize tensors (e.g., embeddings to be stored or weights of a neural network) using distributional quantization. In particular, the API of the package provides two core functions:

  • distquant.quantize(data, k=k) for quantizing a Tensor data of floating point values.
  • distquant.dequantize(quantized_data, **info) for dequantizing quantized_data.

In detail, the API is specified as

import torch
from distquant import quantize, dequantize

data = torch.randn(10_000)

q_data, info = quantize(
    data,
    k=4,              # number of bits to quantize to; alternatively specify `n`. 
    # n=15,           # number of quantization points; mutually exclusive to `k`.
    dist='gaussian',  # distributional assumption
    packbits=True,    # Packs the results into int64, e.g., for `k=4`, there will be 16 values in each element 
                      # for storing. If set to `False`, the indices are returned in raw format.
    beta=1.0,         # Factor hyperparameter for adjusting the range of values (esp. important for small `k`).
    zero=None         # Can enforce a particular zero (by setting it to a resp. float, e.g., `0.`)
)

deq_data = dequantize(data, **info)
  • Bitpacking (packbits=True) allows storing the data efficiently, e.g., 21 3-bit values can be stored in a single int64.
  • beta may be set None (default), which will cause an automatic search for the optimal choice. This can be more expensive but is valuable for smaller k / n.
  • Setting dist=None searches for the best distribution to quantize the provided data.

Supported options for dist:

['gaussian' 'gaussian_cdf', 'logistic', 'logistic_cdf', 'gumbel', 'gumbel_cdf', 'exponential', 'exponential_cdf', 'cauchy_cdf']

📖 Citing

@article{petersen2023distributional,
  title={{Distributional Quantization}},
  author={Petersen, Felix and Sutter, Tobias},
  year={2021--2023}
}

📜 License

distquant is released under the MIT license. See LICENSE for additional details about it.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

distquant-0.1.0.tar.gz (5.7 kB view details)

Uploaded Source

File details

Details for the file distquant-0.1.0.tar.gz.

File metadata

  • Download URL: distquant-0.1.0.tar.gz
  • Upload date:
  • Size: 5.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.6

File hashes

Hashes for distquant-0.1.0.tar.gz
Algorithm Hash digest
SHA256 29e169db977141e5aab8938d8af99ae24f3cfe6a5d1ea491ec6b11bbcd5801b5
MD5 6642269670a1a5616042cdb2c551a68e
BLAKE2b-256 123ed758a5e0aa38ea1db6fbcaebddffee31b3954d46ec26657af650e71c3bc2

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page