tokenizer tool
Project description
Easy-Tokenizer
Description
Try with the following script: easy_tokenizer input_text
Status
todo
Requirements
Python 3.6+
Installation
pip install easy-tokenizer
Usage
todo
Development
To install package and its dependencies, run the following from project root directory:
python setup.py install
To work the code and develop the package, run the following from project root directory:
python setup.py develop
To run unit tests, execute the following from the project root directory:
python setup.py test
0.0.1 (2019-10-22)
Add the first version of the tokenizer
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
easy_tokenizer-0.0.1.tar.gz
(9.3 kB
view hashes)
Built Distribution
Close
Hashes for easy_tokenizer-0.0.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 248259cc3cd6c6a21749cd23cbff43cc5f14876dc9f25cbdf9b47932a74edc47 |
|
MD5 | 1de05d6619dd18b6eec20bb6f3ac285b |
|
BLAKE2b-256 | fae42ec4c8c8a9c80cca527a7c7bb0bbc58b3b92620c0ccbf1f6bcb2fe3bd5f6 |