bitsandbytes
bitsandbytes enables accessible large language models via k-bit quantization for PyTorch. We provide three main features for dramatically reducing memory consumption for inference and training:
- 8-bit optimizers uses block-wise quantization to maintain 32-bit performance at a small fraction of the memory cost.
- LLM.int8() or 8-bit quantization enables large language model inference with only half the required memory and without any performance degradation. This method is based on vector-wise quantization to quantize most features to 8-bits and separately treating outliers with 16-bit matrix multiplication.
- QLoRA or 4-bit quantization enables large language model training with several memory-saving techniques that don't compromise performance. This method quantizes a model to 4-bits and inserts a small set of trainable low-rank adaptation (LoRA) weights to allow training.
The library includes quantization primitives for 8-bit & 4-bit operations, through bitsandbytes.nn.Linear8bitLt and bitsandbytes.nn.Linear4bit and 8-bit optimizers through bitsandbytes.optim module.
System Requirements
bitsandbytes has the following minimum requirements for all platforms:
- Python 3.10+
- PyTorch 2.4+
- Note: While we aim to provide wide backwards compatibility, we recommend using the latest version of PyTorch for the best experience.
Accelerator support:
Note: this table reflects the status of the current development branch. For the latest stable release, see the document in the 0.50.0 tag.
Legend:
🚧 = Planned | 〰️ = Partially Supported | ✅ = Supported | ❌ = Not Supported
| Platform | Accelerator | Hardware Requirements | LLM.int8() | QLoRA 4-bit | 8-bit Optimizers |
|---|---|---|---|---|---|
| 🐧 Linux, glibc >= 2.24 | |||||
| x86-64 | ◻️ CPU | Minimum: AVX2 Optimized: AVX512F, AVX512BF16 |
✅ | ✅ | ✅ |
🟩 NVIDIA GPU cuda |
SM60+ minimum SM75+ recommended |
✅ | ✅ | ✅ | |
🟥 AMD GPU cuda |
CDNA: gfx908, gfx90a, gfx942, gfx950, gfx1250 RDNA: gfx101X, gfx103X, gfx110X, gfx115X, gfx120X |
✅ | ✅ | ✅ | |
🟦 Intel GPU xpu |
Data Center GPU Max Series Arc A-Series (Alchemist) Arc B-Series (Battlemage) |
✅ | ✅ | ✅ | |
🟪 Intel Gaudi hpu |
Gaudi2, Gaudi3 | ✅ | 〰️ | ❌ | |
| aarch64 | ◻️ CPU | ✅ * | ✅ | ✅ | |
🟩 NVIDIA GPU cuda |
SM75+ | ✅ | ✅ | ✅ | |
| 🪟 Windows 11 / Windows Server 2022+ | |||||
| x86-64 | ◻️ CPU | AVX2 | ✅ | ✅ | ✅ |
🟩 NVIDIA GPU cuda |
SM60+ minimum SM75+ recommended |
✅ | ✅ | ✅ | |
🟥 AMD GPU cuda |
CDNA: gfx908, gfx90a RDNA: gfx101X, gfx103X, gfx110X, gfx115X, gfx120X |
✅ | ✅ | ✅ | |
🟦 Intel GPU xpu |
Arc A-Series (Alchemist) Arc B-Series (Battlemage) |
✅ | ✅ | ✅ | |
| arm64 | ◻️ CPU | ✅ | ✅ | ✅ | |
🟩 NVIDIA GPU cuda |
SM121 | ✅ | ✅ | ✅ | |
| 🍎 macOS 14+ | |||||
| arm64 | ◻️ CPU | Apple M1+ | ✅ * | ✅ | ✅ |
⬜ Metal mps |
Apple M1+ | ✅ * | ✅ | 🚧 | |
:book: Documentation
:heart: Sponsors
The continued maintenance and development of bitsandbytes is made possible thanks to the generous support of our sponsors. Their contributions help ensure that we can keep improving the project and delivering valuable updates to the community.
License
bitsandbytes is MIT licensed.
How to cite us
If you found this library useful, please consider citing our work:
QLoRA
@article{dettmers2023qlora,
title={Qlora: Efficient finetuning of quantized llms},
author={Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},
journal={arXiv preprint arXiv:2305.14314},
year={2023}
}
LLM.int8()
@article{dettmers2022llmint8,
title={LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale},
author={Dettmers, Tim and Lewis, Mike and Belkada, Younes and Zettlemoyer, Luke},
journal={arXiv preprint arXiv:2208.07339},
year={2022}
}
8-bit Optimizers
@article{dettmers2022optimizers,
title={8-bit Optimizers via Block-wise Quantization},
author={Dettmers, Tim and Lewis, Mike and Shleifer, Sam and Zettlemoyer, Luke},
journal={9th International Conference on Learning Representations, ICLR},
year={2022}
}
2 maintainers
Release files for bitsandbytes 0.50.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| bitsandbytes-0.50.2-py3-none-win_arm64.whl | Python 3 | none | Windows ARM64 | Details |
| bitsandbytes-0.50.2-py3-none-win_amd64.whl | Python 3 | none | Windows x86-64 | Details |
| bitsandbytes-0.50.2-py3-none-manylinux_2_24_x86_64.whl | Python 3 | none | Linux glibc 2.24+ x86-64 | Details |
| bitsandbytes-0.50.2-py3-none-manylinux_2_24_aarch64.whl | Python 3 | none | Linux glibc 2.24+ ARM64 | Details |
| bitsandbytes-0.50.2-py3-none-macosx_14_0_arm64.whl | Python 3 | none | macOS 14.0+ ARM64 | Details |
Total release size: 107.2 MB
Release files / bitsandbytes-0.50.2-py3-none-win_arm64.whl
| Download URL | bitsandbytes-0.50.2-py3-none-win_arm64.whl |
|---|---|
| Size | 1.1 MB |
| Tags | Python 3 Windows ARM64 |
|
SHA-256 checksum How to use checksums |
8437ab68a04ea56daf1d6ecb54230fb1d88be4b89fe2d79bc399bc0203b487cf
|
|
BLAKE2b-256 checksum How to use checksums |
a25761636c5b11b0a32e505127a6dce6fa8fcbf73978babe8fa37082ab547f1c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 27, 2026.
Transparency logRelease files / bitsandbytes-0.50.2-py3-none-win_amd64.whl
| Download URL | bitsandbytes-0.50.2-py3-none-win_amd64.whl |
|---|---|
| Size | 39.1 MB |
| Tags | Python 3 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
c697963c8fda3dcd0d7ebd9b5211ae4067feef7cd06e0350d4e816a434fe683d
|
|
BLAKE2b-256 checksum How to use checksums |
7282742dc27a1feab90c8f87f2ed14e6d72d05f9e1cf764b4d2ba30aa9b4a2cb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 27, 2026.
Transparency logRelease files / bitsandbytes-0.50.2-py3-none-manylinux_2_24_x86_64.whl
| Download URL | bitsandbytes-0.50.2-py3-none-manylinux_2_24_x86_64.whl |
|---|---|
| Size | 43.1 MB |
| Tags | Linux glibc 2.24+ x86-64 Python 3 |
|
SHA-256 checksum How to use checksums |
55348a9a4a21bfd99cf8c7b32fe67b4030ae5c2a05738e03c1747f65fa6ec283
|
|
BLAKE2b-256 checksum How to use checksums |
a56ee4e8b75716dbe5e50964f070266e06f4e6806ce051bfb97f52ee162b9310
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 27, 2026.
Transparency logRelease files / bitsandbytes-0.50.2-py3-none-manylinux_2_24_aarch64.whl
| Download URL | bitsandbytes-0.50.2-py3-none-manylinux_2_24_aarch64.whl |
|---|---|
| Size | 23.8 MB |
| Tags | Linux glibc 2.24+ ARM64 Python 3 |
|
SHA-256 checksum How to use checksums |
d5772560dd94c4d9c57f50c9b017450a1707f7687bfd4b3dc86f7342aafe721e
|
|
BLAKE2b-256 checksum How to use checksums |
88d5b2cb5b5a9daf7349a02b1af2c49b6a044fda2702c9cc5dc296f648358327
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 27, 2026.
Transparency logRelease files / bitsandbytes-0.50.2-py3-none-macosx_14_0_arm64.whl
| Download URL | bitsandbytes-0.50.2-py3-none-macosx_14_0_arm64.whl |
|---|---|
| Size | 123.5 kB |
| Tags | Python 3 macOS 14.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
4311f52a880b341bada639e4edd1a3c8d786830c9c93cdde29eaa1f062c8f8e5
|
|
BLAKE2b-256 checksum How to use checksums |
55bf5290208ce1ecf0f2e6a916fc72a75f6e68021ecfd69e7014fc95998532eb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 27, 2026.
Transparency log
