Distributional Quantization
Project description
Distributional Quantization
Distributional quantization provides an information theoretically optimal and efficient quantization of data following a distributional prior.
This repository includes a pre-print of an excerpt of the distributional quantization paper.
Simulation plots can be found here.
The distquant
package requires PyTorch and can be installed via
pip install distquant
💻 Usage
The distquant
package can be used to efficiently and effectively quantize tensors (e.g., embeddings to be stored or weights of a neural network) using distributional quantization.
In particular, the API of the package provides two core functions:
distquant.quantize(data, k=k)
for quantizing a Tensordata
of floating point values.distquant.dequantize(quantized_data, **info)
for dequantizingquantized_data
.
In detail, the API is specified as
import torch
from distquant import quantize, dequantize
data = torch.randn(10_000)
q_data, info = quantize(
data,
k=4, # number of bits to quantize to; alternatively specify `n`.
# n=15, # number of quantization points; mutually exclusive to `k`.
dist='gaussian', # distributional assumption
packbits=True, # Packs the results into int64, e.g., for `k=4`, there will be 16 values in each element
# for storing. If set to `False`, the indices are returned in raw format.
beta=1.0, # Factor hyperparameter for adjusting the range of values (esp. important for small `k`).
zero=None # Can enforce a particular zero (by setting it to a resp. float, e.g., `0.`)
)
deq_data = dequantize(data, **info)
- Bitpacking (
packbits=True
) allows storing the data efficiently, e.g., 21 3-bit values can be stored in a single int64. beta
may be setNone
(default), which will cause an automatic search for the optimal choice. This can be more expensive but is valuable for smallerk
/n
.- Setting
dist=None
searches for the best distribution to quantize the provided data.
Supported options for dist
:
['gaussian' 'gaussian_cdf', 'logistic', 'logistic_cdf', 'gumbel', 'gumbel_cdf', 'exponential', 'exponential_cdf', 'cauchy_cdf']
📖 Citing
@article{petersen2023distributional,
title={{Distributional Quantization}},
author={Petersen, Felix and Sutter, Tobias},
year={2021--2023}
}
📜 License
distquant
is released under the MIT license. See LICENSE for additional details about it.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file distquant-0.1.0.tar.gz
.
File metadata
- Download URL: distquant-0.1.0.tar.gz
- Upload date:
- Size: 5.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 29e169db977141e5aab8938d8af99ae24f3cfe6a5d1ea491ec6b11bbcd5801b5 |
|
MD5 | 6642269670a1a5616042cdb2c551a68e |
|
BLAKE2b-256 | 123ed758a5e0aa38ea1db6fbcaebddffee31b3954d46ec26657af650e71c3bc2 |