Tokenizer evaluation library
Project description
tokeval
A toolkit for evaluating the quality of the text tokenizers.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tokenwise-0.1.0.tar.gz
(7.9 kB
view hashes)
Built Distribution
Close
Hashes for tokenwise-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a95867d1cf325ae55d0b3608d420934392224c16de70a33e57893fdcfa8c1a37 |
|
MD5 | 80296f2bc8b012b465743778e2af1e14 |
|
BLAKE2b-256 | e37c0042cf93737f13801f00368fbd9a8b62ec152d5424419b35016ea79d98d6 |