Skip to main content

k-bit optimizers and matrix multiplication routines.

Project description

bitsandbytes

Downloads Downloads Downloads

The bitsandbytes library is a lightweight Python wrapper around CUDA custom functions, in particular 8-bit optimizers, matrix multiplication (LLM.int8()), and 8 & 4-bit quantization functions.

The library includes quantization primitives for 8-bit & 4-bit operations, through bitsandbytes.nn.Linear8bitLt and bitsandbytes.nn.Linear4bit and 8-bit optimizers through bitsandbytes.optim module.

There are ongoing efforts to support further hardware backends, i.e. Intel CPU + GPU, AMD GPU, Apple Silicon, hopefully NPU.

Please head to the official documentation page:

https://huggingface.co/docs/bitsandbytes/main

License

bitsandbytes is MIT licensed.

We thank Fabio Cannizzo for his work on FastBinarySearch which we use for CPU quantization.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

bitsandbytes-0.45.5-py3-none-win_amd64.whl (75.4 MB view details)

Uploaded Python 3Windows x86-64

bitsandbytes-0.45.5-py3-none-manylinux_2_24_x86_64.whl (76.1 MB view details)

Uploaded Python 3manylinux: glibc 2.24+ x86-64

File details

Details for the file bitsandbytes-0.45.5-py3-none-win_amd64.whl.

File metadata

File hashes

Hashes for bitsandbytes-0.45.5-py3-none-win_amd64.whl
Algorithm Hash digest
SHA256 ed1c61b91d989d6a33fd05737d6edbf5086d8ebc89235ee632c7a19144085da2
MD5 540374c1f7f98dc78de9008d536a5b13
BLAKE2b-256 a64c77b535e025ce780d2ada8271c1e481fb7337c1df2588a52fe1c9bd87d2e8

See more details on using hashes here.

Provenance

The following attestation bundles were made for bitsandbytes-0.45.5-py3-none-win_amd64.whl:

Publisher: python-package.yml on bitsandbytes-foundation/bitsandbytes

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file bitsandbytes-0.45.5-py3-none-manylinux_2_24_x86_64.whl.

File metadata

File hashes

Hashes for bitsandbytes-0.45.5-py3-none-manylinux_2_24_x86_64.whl
Algorithm Hash digest
SHA256 a5453f30cc6aab6ccaac364e6bf51a7808d3da5f71763dffeb6d9694c59136e4
MD5 f06c3f02ead0aa9e973573e26f9efca9
BLAKE2b-256 07b7cb5ce4d1a382cf53c19ef06c5fc29e85f5e129b4da6527dd207d90a5b8ad

See more details on using hashes here.

Provenance

The following attestation bundles were made for bitsandbytes-0.45.5-py3-none-manylinux_2_24_x86_64.whl:

Publisher: python-package.yml on bitsandbytes-foundation/bitsandbytes

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page