Last released Feb 6, 2025
A fast WordPiece tokenizer implementation in Rust with Python bindings
Supported by