Unicode-based language-agnostic (over-) tokenizer.
Project description
This package provides a simple language-agnostic tokenizer.
Copyright ® 2016, Luís Gomes <luismsgomes@gmail.com>.
This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for overtokenizer-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee1d5390be048364f55468e50c050d99de8618d29217327bdc748686bc863e65 |
|
MD5 | 226296a151415e6ae5e931479512e993 |
|
BLAKE2b-256 | 09a28374091dcf73c6ce3bbdabf02cd3869443dc9695d9ae9cf111f06e590be2 |