k-bit optimizers and matrix multiplication routines.
Project description
bitsandbytes
The bitsandbytes library is a lightweight Python wrapper around CUDA custom functions, in particular 8-bit optimizers, matrix multiplication (LLM.int8()), and 8 & 4-bit quantization functions.
The library includes quantization primitives for 8-bit & 4-bit operations, through bitsandbytes.nn.Linear8bitLt and bitsandbytes.nn.Linear4bit and 8-bit optimizers through bitsandbytes.optim module.
There are ongoing efforts to support further hardware backends, i.e. Intel CPU + GPU, AMD GPU, Apple Silicon. Windows support is quite far along and is on its way as well.
Please head to the official documentation page:
https://huggingface.co/docs/bitsandbytes/main
License
The majority of bitsandbytes is licensed under MIT, however small portions of the project are available under separate license terms, as the parts adapted from Pytorch are licensed under the BSD license.
We thank Fabio Cannizzo for his work on FastBinarySearch which we use for CPU quantization.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for bitsandbytes-0.43.1-py3-none-win_amd64.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 52c1c7189a6ca006555a9663e544e75f40520a97a26e075411f9f9aca0771fcd |
|
| MD5 | b8a3fd842bb87409c8df2fcb05e7fe78 |
|
| BLAKE2b-256 | 32e8ab6c97347c99cf5d18d0750c7336270719b17cb0610eb0a44cf833aa378f |
Hashes for bitsandbytes-0.43.1-py3-none-manylinux_2_24_x86_64.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | a81c826d576d6d691c7b4a7491c8fdc0f37f769795d6ca2e54afa605d2c260a3 |
|
| MD5 | ce47198eb1fff8b7210fe3e3b34fabe1 |
|
| BLAKE2b-256 | 2fa4d8c8c1f69ceb3afdc285d62c65bec8d46900d70e81c9a8b24883001e23f8 |
