This release is a pre-release and may not be stable for production use.
bitsandbytes2
Quantization kernels, an inference runtime, and a coding agent in one repository. This is the working branch. A history-purged private orphan comes later from a cleaned tip of this tree.
| Path | Product | Install name |
|---|---|---|
. (bitsandbytes/, csrc/) |
VQ-L2 / NF4 / int8 kernels | bitsandbytes-kernels (import bitsandbytes) |
bitsandbytes.cpp/ |
Hugging Face inference runtime | bitsandbytes-cpp |
bitsandbytes-code/ |
coding agent | bitsandbytes-code |
| metapackage | pulls the three | bitsandbytes2 |
See ECOSYSTEM.md for layout, sources, and what is still dirty.
Install (target)
No compiler on the user machine. One command pulls the three packages:
pip install bitsandbytes2
Individual pieces stay installable (bitsandbytes-kernels, bitsandbytes-cpp,
bitsandbytes-code). First public index is TestPyPI. From this checkout:
pip install -e .
installs the Python package only. CUDA kernels still need a cmake build in
this development tree; release wheels will ship the .so / .dylib.
What this package is
- VQ-L2 vector quantization (2–5 bits) for weights and KV cache
- Fused kernels (
quantize_vq_l2,fused_vq_l2_matmul_small_m, NVFP4) - C ABI (
csrc/bnb_backend_abi.h) consumed bybitsandbytes.cpp - Dynamic bit allocation (
dynamic_bitalloc.py)
CUDA and CPU binaries exist. Metal sources exist (csrc/backend_api_metal.mm);
a release dylib is not in this tree yet. AMD is later.
The coding agent currently needs PostgreSQL at runtime.
License
MIT. See LICENSE and NOTICE.md.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file bitsandbytes_kernels-0.50.0.dev0-cp311-cp311-manylinux_2_24_x86_64.whl.
File metadata
- Download URL: bitsandbytes_kernels-0.50.0.dev0-cp311-cp311-manylinux_2_24_x86_64.whl
- Upload date:
- Size: 15.6 MB
- Tags: CPython 3.11, manylinux: glibc 2.24+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.11.8
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
28638ba6a3df685d1c74d8ed4442688532161b886909ee93457514f69165dfa2
|
|
| MD5 |
22c157cbba76a5a3b479da36286f34d9
|
|
| BLAKE2b-256 |
d652f58eb5b071ff4d3badfdd2c1a6f13f952f9560c07fe6d52fc6663b638fb5
|
