This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).
Project description
The author of this package has not provided a project description
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
python-ucto-0.6.1.tar.gz
(78.8 kB
view hashes)
Built Distributions
Close
Hashes for python_ucto-0.6.1-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bfdfa2ab75455465b3966f800642cb18ccd48360701556bf6f2e212642239143 |
|
MD5 | a981becc57c279993216d4998c76de00 |
|
BLAKE2b-256 | 46fe8aba6cd2c784d5c8e9ad29ccbe8cfa3734ba6e442aaffdf501ebe2e79bf5 |
Close
Hashes for python_ucto-0.6.1-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4ac34c266a57862a78cf37f385fabd6dcd422ad75b9c25896ff971d5b1321edd |
|
MD5 | 64c635b9952ca99da274bbc76099da90 |
|
BLAKE2b-256 | f535a3643b5b62b969191d7f6688f6a5b0dfe1d1364b65fe5673851c2a75a7a6 |
Close
Hashes for python_ucto-0.6.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3f487ac5f047253ce87566eef4f89842285a13bbdaa9445a8ad1b717e28478c9 |
|
MD5 | 8c127c9dfc7d1457a64e33149068215f |
|
BLAKE2b-256 | c8ac3ec587ae4f285c641d2abc0fd1962b289cd370903193d794614ecd8843b9 |
Close
Hashes for python_ucto-0.6.1-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7874af7a6f58c4f2e944f99229967055ad7f868e043c3c465c16e8fd1a65c219 |
|
MD5 | e731636891a0278261340fbe57504a97 |
|
BLAKE2b-256 | 014cb46b59d41c697c9335f220b147949e641c6eb7aa0e2327e8fa605fba26b2 |
Close
Hashes for python_ucto-0.6.1-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cb61c650a9f8bac89b62b6975ecd11e7892bc188c6f762f8bc8ffaf8ab00142c |
|
MD5 | 765fa8579d188e5f95b2ad8084c70538 |
|
BLAKE2b-256 | 8c2d76439a0d2a06ec94115532cad1d249ae5fde98dea7741e7dea9bcf53e814 |
Close
Hashes for python_ucto-0.6.1-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e6c0e1150b2c876d3b597c7b62d194220b3560cb5ee7c97ee2d8a630c345a668 |
|
MD5 | e068e131d19967d028ebab6786aba4b8 |
|
BLAKE2b-256 | 70ca34d468d7156743cb93b5554aaafad6b72ba989c76afc466b7fb100be5737 |
Close
Hashes for python_ucto-0.6.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 71b25c964f87b49f4e775d09f77f43446303bac3bc9b34b3fc4146d3f4d2848d |
|
MD5 | 594aeca83c9c7604c48c80719c29a999 |
|
BLAKE2b-256 | 176dbaac482ae79393ac72708a1ac73445ebc956dec555925d511bd41012bfe1 |
Close
Hashes for python_ucto-0.6.1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 567b652b85359c8415def374b316589265a37b4fc3e9b919f81cd2fabe83a6fd |
|
MD5 | f6d9c084503da7fa8102ac58af834018 |
|
BLAKE2b-256 | 81f0ed4480820515386f88452e4442ac8f95d23107dc395ce1265e75973a8bd3 |
Close
Hashes for python_ucto-0.6.1-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8fc634a82d914611e3a7f2c7e7923fece4e07e8c91af33512660958b0828d82 |
|
MD5 | e36fcd32e2e1b740fd6150960c4b2dda |
|
BLAKE2b-256 | 6c0e2f5f834c0431defa07c516bde4a32729978da587042b1adf6b1a1038f1ed |
Close
Hashes for python_ucto-0.6.1-cp39-cp39-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | edc747b14350429c4e537f6f60f400f1d5eea82c7379ad1f7f26da6bdf199e96 |
|
MD5 | b3634bbe2280755d847a8d8aa9722ca5 |
|
BLAKE2b-256 | 7f6301c1debbbc6e805ef87e962e4276ee1d00aedf7188489234c6d959776368 |
Close
Hashes for python_ucto-0.6.1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1732d671db006ebfd2aeb206cc6c6359d91d7272d142bd14a1ad33b18cbaa101 |
|
MD5 | 56a36e83f91dd8ad03dde21434e00e7c |
|
BLAKE2b-256 | 4cbe4d3b5402ac7910027cf62d8df600b33bcef2cbf6b60aab6f23f741e6200f |
Close
Hashes for python_ucto-0.6.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d8829f1ff6c8579b20c0a568fac3cf5ebf0242796fb02f31ea8ceab4cd97d1ce |
|
MD5 | e4ae2f1fa1bcf57c57634ecdac8727ed |
|
BLAKE2b-256 | 0e1f1a4c8a33beb468b62232f3acf43e2b5cd3a7d469fe3435a229a4c593e7c7 |
Close
Hashes for python_ucto-0.6.1-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 51a962f062357f78428ce694660513795d7604d1d63f7a0c2f57c9979ac80aa0 |
|
MD5 | 0455f895bcac43b1c24846518ef9835f |
|
BLAKE2b-256 | 871c3e375923de0f3d3e74d42094e221f911f944389124a947a37df04ec023c7 |
Close
Hashes for python_ucto-0.6.1-cp38-cp38-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4a5b18872781d7b9611590b1237a0016bca7fda050c6784c9e8dbedde7432180 |
|
MD5 | f136bb31aa6f3546622b3f109d7948d4 |
|
BLAKE2b-256 | d1f5a0462b4af132f0c74baa9b294f731fd2d4cc6b35b0fe423374bb508c7df2 |
Close
Hashes for python_ucto-0.6.1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 12b9ca9194b6e9983bbae91302f5189bb15d3a889e86bf24f94597b0b1da0f1a |
|
MD5 | 5ab0bc528fcc01061b366f608b7692ea |
|
BLAKE2b-256 | 1b58ba4e3a0384a3b8f4cf33a5455baabd0bde89e0faaa20cedef77a5ddd38ac |
Close
Hashes for python_ucto-0.6.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4fb4051c8da95e90bd66d923fdbe3da6ef896608201695dbe33e9926e33c686b |
|
MD5 | 8885e1d89c0f4937d76c7e832b470a64 |
|
BLAKE2b-256 | 0b17a4bccd37bf1bced0d3142483a509f99fa09cbb7ea25e582c9a7f962064fc |
Close
Hashes for python_ucto-0.6.1-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6d3d42b702c851bc0cc354c790545de9cf93612265e56d6ec26dfba3bc2b3a2e |
|
MD5 | 3810dfd96f7647a4d3668a46f543d879 |
|
BLAKE2b-256 | 6d938dc25a6df600ebe9348ac7e931208833f30190458ea5d0917876b7f3f8da |
Close
Hashes for python_ucto-0.6.1-cp37-cp37m-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5c8e39c7e894f698ddf705e9d38c82ba90e0514ad2d3bc6d4bba02a4f48e9322 |
|
MD5 | a6c01ff2da487ee60153424eac02f600 |
|
BLAKE2b-256 | 990aeea63398ae238dfb02203f8da89eb8e5807271a7aedc5f9b154acd10df71 |
Close
Hashes for python_ucto-0.6.1-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c9dc49103f68c38bf800509c18790bd332c2ff7b81f2727587c8deed0ca7803e |
|
MD5 | 40991cd59596ac97e8124dab62b04d85 |
|
BLAKE2b-256 | a885ee85ccb6add94fd90282396e9fde2d675f323296ebe06190d3692d34bbe2 |
Close
Hashes for python_ucto-0.6.1-cp36-cp36m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f5cfcef561f1311da0c7f858ab2e5197be3c923fb96f005e2a27ba89421d19d8 |
|
MD5 | 39256c8118a3db4e6633ea5d5299f63f |
|
BLAKE2b-256 | 275bfa49f125a4d8de5058c7225730c2b15d44e60e6260c8c9869851e1642387 |
Close
Hashes for python_ucto-0.6.1-cp36-cp36m-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f6045873618a516b0011cf95acfaa176fa9ca7c7b31c004aa3b84ef88d68bc80 |
|
MD5 | aa7a38e6d651722ed314f469788fcea4 |
|
BLAKE2b-256 | f9b4e7c80667ff6d70e1cc931e18bb79268f8696380bcaa3207c1fd0ada00561 |
Close
Hashes for python_ucto-0.6.1-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | df302709623acd46457da7a8d889844a0a13a17135f4b17330619ce2ccd557f4 |
|
MD5 | a78d7fee08bc7623d32e376cb4504173 |
|
BLAKE2b-256 | 91d08a160fb805a9848c4f9e764f266bfc5297396a6af6e65b9307611166e1e8 |