This release is a pre-release and may not be stable for production use.
bitsandbytes2
Quantization kernels, an inference runtime, and a coding agent in one repository. This is the working branch. A history-purged private orphan comes later from a cleaned tip of this tree.
| Path | Product | Install name |
|---|---|---|
. (bitsandbytes/, csrc/) |
VQ-L2 / NF4 / int8 kernels | bitsandbytes-kernels (import bitsandbytes) |
bitsandbytes.cpp/ |
Hugging Face inference runtime | bitsandbytes-cpp |
bitsandbytes-code/ |
coding agent | bitsandbytes-code |
| metapackage | pulls the three | bitsandbytes2 |
See ECOSYSTEM.md for layout, sources, and what is still dirty.
Install (target)
No compiler on the user machine. One command pulls the three packages:
pip install bitsandbytes2
Individual pieces stay installable (bitsandbytes-kernels, bitsandbytes-cpp,
bitsandbytes-code). First public index is TestPyPI. From this checkout:
pip install -e .
installs the Python package only. CUDA kernels still need a cmake build in
this development tree; release wheels will ship the .so / .dylib.
What this package is
- VQ-L2 vector quantization (2–5 bits) for weights and KV cache
- Fused kernels (
quantize_vq_l2,fused_vq_l2_matmul_small_m, NVFP4) - C ABI (
csrc/bnb_backend_abi.h) consumed bybitsandbytes.cpp - Dynamic bit allocation (
dynamic_bitalloc.py)
CUDA and CPU binaries exist. Metal sources exist (csrc/backend_api_metal.mm);
a release dylib is not in this tree yet. AMD is later.
The coding agent currently needs PostgreSQL at runtime.
License
MIT. See LICENSE and NOTICE.md.
1 maintainer
Metadata
1 maintainer
Release files for bitsandbytes-kernels 0.50.0.dev0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| bitsandbytes_kernels-0.50.0.dev0-cp311-cp311-manylinux_2_24_x86_64.whl | CPython 3.11 | CPython 3.11 | Linux glibc 2.24+ x86-64 | Details |
Release files / bitsandbytes_kernels-0.50.0.dev0-cp311-cp311-manylinux_2_24_x86_64.whl
| Download URL | bitsandbytes_kernels-0.50.0.dev0-cp311-cp311-manylinux_2_24_x86_64.whl |
|---|---|
| Size | 15.6 MB |
| Tags | CPython 3.11 Linux glibc 2.24+ x86-64 |
|
SHA-256 checksum How to use checksums |
28638ba6a3df685d1c74d8ed4442688532161b886909ee93457514f69165dfa2
|
|
BLAKE2b-256 checksum How to use checksums |
d652f58eb5b071ff4d3badfdd2c1a6f13f952f9560c07fe6d52fc6663b638fb5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.8
|
