Tools for tokenizer develope and evaluation
Project description
Tokenizer Tools
Tools/Utils for NLP (including dataset reading, tagset encoding & decoding, metrics computing) | NLP 工具集(包含数据集读取、tagset 编码和解码、指标的计算等)
Free software: MIT license
Documentation: https://tokenizer-tools.readthedocs.io.
Features
常见数据集格式的读取
多种 Tagset 的编码和解码
指标的计算
TODO
改变项目的名字,tokenizer_tools 已经无法正确描述现在项目的功能
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
History
0.1.0 (2018-09-05)
First release on PyPI.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tokenizer_tools-0.13.1.tar.gz
(24.4 kB
view hashes)
Built Distribution
Close
Hashes for tokenizer_tools-0.13.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b32a11f77efbcb89b9eb414da1a2f5bd5d5c034caeece92f0b20a22e848640ce |
|
MD5 | 575ce4f1b4583b1f68fb1e2d50ab0b1f |
|
BLAKE2b-256 | b09c8be7c373f8f6f9137a57e835a9e943c72604bcbd43afc537bbb1b2d3246a |