This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).
Maintainers
Project description
The author of this package has not provided a project description
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
| Filename, size & hash SHA256 hash help | File type | Python version | Upload date |
|---|---|---|---|
| python-ucto-0.4.7.tar.gz (3.9 kB) Copy SHA256 hash SHA256 | Source | None | May 16, 2018 |
