Skip to main content

k-bit optimizers and matrix multiplication routines.

Project description

bitsandbytes

Downloads Downloads Downloads

The bitsandbytes library is a lightweight Python wrapper around CUDA custom functions, in particular 8-bit optimizers, matrix multiplication (LLM.int8()), and 8 & 4-bit quantization functions.

The library includes quantization primitives for 8-bit & 4-bit operations, through bitsandbytes.nn.Linear8bitLt and bitsandbytes.nn.Linear4bit and 8-bit optimizers through bitsandbytes.optim module.

There are ongoing efforts to support further hardware backends, i.e. Intel CPU + GPU, AMD GPU, Apple Silicon. Windows support is quite far along and is on its way as well.

Please head to the official documentation page:

https://huggingface.co/docs/bitsandbytes/main

bitsandbytes multi-backend alpha release is out!

🚀 Big news! After months of hard work and incredible community contributions, we're thrilled to announce the bitsandbytes multi-backend alpha release! 💥

Now supporting:

  • 🔥 AMD GPUs (ROCm)
  • Intel CPUs & GPUs

We’d love your early feedback! 🙏

👉 Instructions for your pip install here

We're super excited about these recent developments and grateful for any constructive input or support that you can give to help us make this a reality (e.g. helping us with the upcoming Apple Silicon backend or reporting bugs). BNB is a community project and we're excited for your collaboration 🤗

License

bitsandbytes is MIT licensed.

We thank Fabio Cannizzo for his work on FastBinarySearch which we use for CPU quantization.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

bitsandbytes-0.45.2-py3-none-win_amd64.whl (69.1 MB view details)

Uploaded Python 3 Windows x86-64

bitsandbytes-0.45.2-py3-none-manylinux_2_24_x86_64.whl (69.7 MB view details)

Uploaded Python 3 manylinux: glibc 2.24+ x86-64

File details

Details for the file bitsandbytes-0.45.2-py3-none-win_amd64.whl.

File metadata

File hashes

Hashes for bitsandbytes-0.45.2-py3-none-win_amd64.whl
Algorithm Hash digest
SHA256 e1893170455422924156760bae9e210ae26e8bd2ce7b21065d24b47decbe6963
MD5 4feea84b80d1dd1f55caab1645826e1d
BLAKE2b-256 cb33550bcfe84f08ee20f3bcc1b129dcadaf7f2d1a065ce9812476fc7be2874a

See more details on using hashes here.

Provenance

The following attestation bundles were made for bitsandbytes-0.45.2-py3-none-win_amd64.whl:

Publisher: python-package.yml on bitsandbytes-foundation/bitsandbytes

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file bitsandbytes-0.45.2-py3-none-manylinux_2_24_x86_64.whl.

File metadata

File hashes

Hashes for bitsandbytes-0.45.2-py3-none-manylinux_2_24_x86_64.whl
Algorithm Hash digest
SHA256 ba3a720187f518b172ebce4081049c682ae3fd8284947e22499b256ff99a2bc3
MD5 669833ef27dee18f9819cb141ce68899
BLAKE2b-256 db9d9382259196d7ad7f3550702390081224e673a705e75b5660ee377b592fc0

See more details on using hashes here.

Provenance

The following attestation bundles were made for bitsandbytes-0.45.2-py3-none-manylinux_2_24_x86_64.whl:

Publisher: python-package.yml on bitsandbytes-foundation/bitsandbytes

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page