Last released Apr 26, 2026
Entropy-optimal tokenizer — faster and denser than BPE, lossless, GIL-free Rust backend
Supported by