Bltzr (pronounced as Baltazar) is a byte-level tokenizer
Project description
BaLTaZaR -- byte-level tokenizer
bltzr package provides the Tokenizer class, which is a a byte-level tokenizer for LLMs,
plus the SqlDataset class for working with datasets in PostgreSQL databases.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
bltzr-0.2.5.tar.gz
(16.0 kB
view hashes)
Built Distribution
bltzr-0.2.5-py2.py3-none-any.whl
(16.7 kB
view hashes)
Close
Hashes for bltzr-0.2.5-py2.py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | def55a8a564557209ff80c003a4a32df40ebe392b58c25784d73150dec19dedd |
|
| MD5 | 48f5e3833816bf8fea749f405ab163dc |
|
| BLAKE2b-256 | bd4455692e1d5295964d604d1411f379564a391385c5dc7931d62e95c4ed7dd8 |
