https://pypi.org/project/tokscope/ Recent updates to the Python Package Index for tokscope en https://pypi.org/project/tokscope/0.1.0/ Measure how efficiently LLM tokenizers handle your language: fertility, UNK rate, and token premium across HuggingFace, tiktoken, and SentencePiece tokenizers karurpabe@gmail.com Fri, 17 Jul 2026 19:17:21 GMT