Last released Apr 3, 2025
A light-weight & fast library for Byte Pair Encoding (BPE) tokenization.
Supported by