Last released Aug 27, 2026
A production-grade, from-scratch implementation of a Byte-Pair Encoding (BPE) tokenizer.