Last released Sep 4, 2026
Unicode-aware, streaming BPE training and tiktoken-compatible encoding
Last released Jul 27, 2026
Deprecated: use ffbpe for Unicode-aware, streaming BPE training and encoding
Last released Apr 23, 2026
Safe parser-based PyTorch checkpoint converter to safetensors