Implementation of the BitLinear layer from: The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Project description
bitlinear-pytorch
Implementation of the BitLinear layer from: The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Install
pip install bitlinear-pytorch
Usage
import torch
from bitlinear_pytorch import BitLinear, replace_linear_with_bitlinear
class TinyMLP(nn.Module):
def __init__(self):
super(TinyMLP, self).__init__()
self.layers = nn.Sequential(
nn.Linear(784, 256),
nn.ReLU(),
nn.Linear(256, 128),
nn.ReLU(),
nn.Linear(128, 10),
)
def forward(self, x):
return self.layers(x)
model = TinyMLP()
replace_linear_with_bitlinear(model)
# or use BitLinear directly
bitlinear = BitLinear(784, 256)
License
MIT
Citation
@misc{ma2024era,
title={The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits},
author={Shuming Ma and Hongyu Wang and Lingxiao Ma and Lei Wang and Wenhui Wang and Shaohan Huang and Li Dong and Ruiping Wang and Jilong Xue and Furu Wei},
year={2024},
eprint={2402.17764},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
TODO
- Implement base BitLinear layer
- Add example usage
- Setup Github Actions workflow
- Implement memory efficient weight encoding/decoding
- Implement Fast Inference (CUDA/CPU/VHDL)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for bitlinear_pytorch-0.4.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fad40ed588eff6c325689fa77f53046e065e53652638ae3fd5a623ff3d215199 |
|
MD5 | 77381f0b9ac0e6cc90acc71ec7be2bda |
|
BLAKE2b-256 | 59896ec86fce92e7cd7066de7baec1623a9ad0ad9c37df0e126c2bb18fb8be87 |