Last released Jun 14, 2026
BPE tokenizer for Indic scripts (Devanagari, Bengali, Tamil, Telugu, and more)
Supported by