This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).
Project description
The author of this package has not provided a project description
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
python-ucto-0.6.2.tar.gz
(79.0 kB
view hashes)
Built Distributions
Close
Hashes for python_ucto-0.6.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 255a3a39c4aa645880815d0df2572d4b5f661ee3bc9add32cb0c11a77a308484 |
|
MD5 | bbbdf05b18a322aa0dce1560095fa848 |
|
BLAKE2b-256 | 585f94ee0e27ef2bc8b3587d1f1b51f29be1203a8a5e1a320b34593ae397943b |
Close
Hashes for python_ucto-0.6.2-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f9b24cd4a78ae948c6736931a7166c14641cef2a16e763c7401c6a04dfd5ddc6 |
|
MD5 | 4995ae3dc56b807d3bc45fa407cbe6fe |
|
BLAKE2b-256 | 8b3798b195283d8357383e57ac36d593f0a667ea5cb3946d92f7e603feac8b90 |
Close
Hashes for python_ucto-0.6.2-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a5fc01229ddd54148a15e56c33aea8404d25b37bd3ac3c503b5bd73b69afa2a9 |
|
MD5 | 691d359e725f1a15e75fd142dbf85434 |
|
BLAKE2b-256 | cdda88359f5b1eb939a75411cba1f2340c4798bef98c6575bca6727b6dbe6002 |
Close
Hashes for python_ucto-0.6.2-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ba7653bbd0a34a0ed6c2a3da3f0e70816f06128d7143a8f2c7e2428578acabc3 |
|
MD5 | 1f49bfbc4ca4dcc9a8b3b68cc1ff05be |
|
BLAKE2b-256 | 782445cd382d13bfa95ee72150a29aefd7778c5c4d94d82afc712002ad2c07c2 |
Close
Hashes for python_ucto-0.6.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bbe16211624a6b06bbff747c57790e8c9c40a7aeb4c5f2c57132793e283fa4b7 |
|
MD5 | 76f554e03843c54f0689d5499882f1e1 |
|
BLAKE2b-256 | 8a78f217c5e4c860dee9d83686ea159641cfd667bc1288afb295cacc271a05fb |
Close
Hashes for python_ucto-0.6.2-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 711fef4c8c216ebcbfdaf46be4decd2c68e474469e45b1298a400721e5ede301 |
|
MD5 | 8bf81483d6ba686ef6efdd7f5401860a |
|
BLAKE2b-256 | 3ecd3dcdbdb8ad1fe7ba3a2064649ec0cbbb5a9244a7be3f7ea990829ca45208 |
Close
Hashes for python_ucto-0.6.2-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f717936e13d1fae9bd5491bcd3c73d8beee3cb200b73df700f36f0f26b0ccae2 |
|
MD5 | 4688b1d3a790c1d9bf1096c73ae1cd7f |
|
BLAKE2b-256 | 5347a61000476830f3c3770861daab63bad9cdd1f4253c6bdf88294771b37194 |
Close
Hashes for python_ucto-0.6.2-cp39-cp39-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e9f83491c757d4bee0ec978c2355753fd8465a48bc7cd231d6d8b4bb9673ee68 |
|
MD5 | 49a5ebd8dc9507494c7e4216b1d2f363 |
|
BLAKE2b-256 | af7af8029e0cbc84177cc5c189bc68f5fc59a00e020103d6fcf6e63d9e1c25b1 |
Close
Hashes for python_ucto-0.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b5c16d1068c9c419082b57e3a95086dcb427d1f97de689f75691378bc7ba9597 |
|
MD5 | df293a853d2e7f0c63d84527672d0e24 |
|
BLAKE2b-256 | 2804e7f4148b8781493520571dd15ce7fe21565362d3c0488aefbe1d5d4ed4f4 |
Close
Hashes for python_ucto-0.6.2-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4af5a29efa8dbc97841d1192ac0a702864ddb056b57959fe358c18c526bde4db |
|
MD5 | d152bfd6fe50ec7c29fd4ab33b460e6d |
|
BLAKE2b-256 | e1922e42f190d9c3e30c476ce4a78931c3c01753f0a92cbe179f68765343decf |
Close
Hashes for python_ucto-0.6.2-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0746cb14c20c430b2b2b806cde467edaf29b7b7406845b345d1765e11511e7eb |
|
MD5 | 7a987fc4213141e383a1810be451423c |
|
BLAKE2b-256 | c5101376d1db9e53ba14ffe67b2eac2f6efd2babf6286a586e8eef5d14e061d8 |
Close
Hashes for python_ucto-0.6.2-cp38-cp38-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eefaf5cd6d4066b3a593b5534deff3f25fe0bf544916d3eb61c6678cea7c0fd7 |
|
MD5 | eba79b413bc1819a087a25ba6b33593c |
|
BLAKE2b-256 | 8c97d4df46ad386e25e0808117bf0b4aac6ee007c3ef24eaa756b20a1ca1ee93 |
Close
Hashes for python_ucto-0.6.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b95a12bf9ded5a145ab1d54d0d777c3392f868f6e0db4c5057506249eae44d8e |
|
MD5 | aba81b09bb76cb09d8ee147ef4abe45d |
|
BLAKE2b-256 | 0f74edcb002bb735184e5441a6faa58490c60b9f0b374afd0984c1fb93e8be51 |
Close
Hashes for python_ucto-0.6.2-cp38-cp38-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 79894bdf139c3e72d2c0015f0a521adca0b823856eca3300b08f6c79e78b2d5b |
|
MD5 | 7a23c23dcc1b49a03da34dfca4d665be |
|
BLAKE2b-256 | 603226b1119231f278e2c1eba6d58180aa6ed315608db06cde06ea747fa608d0 |
Close
Hashes for python_ucto-0.6.2-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 82a08343470ff7332fcdaedec3a1b7740601563a23f283d98e7ec183a70e57f3 |
|
MD5 | 69cb82778f403caf27e562ce3f8fc854 |
|
BLAKE2b-256 | cbbb53fb48e4e2301c36098a5818940da2f905081f93839bfac284151754b05b |
Close
Hashes for python_ucto-0.6.2-cp37-cp37m-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dffb7793936d1fa4cfba727c0db165ce1babeb50baac6f2e88225ec4b8d85174 |
|
MD5 | 8f9c9c0bc88dfa0e551290b831911e75 |
|
BLAKE2b-256 | 8bdd60851b5c80dc18805f2426457dbd5ac0a624a79272212436d3676d3c8b8d |
Close
Hashes for python_ucto-0.6.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4888a14d74fa4c51e5340c644622315609a56017d8dce90d4e130458ec0a3b6a |
|
MD5 | b83df5fd11fac5e713a3d58a32c7a090 |
|
BLAKE2b-256 | 3623e8c980ff086caf20e8a2d56477997091d85e57df2fedc20c3d15b0fd0935 |
Close
Hashes for python_ucto-0.6.2-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e67ef5ac3430a95834b32131caccf75a5ef7180aba78eb5b86e6ab98a7e02bbc |
|
MD5 | 91abd533bc57b8bf7c5d434960faf1e8 |
|
BLAKE2b-256 | 38575ae160ace7af46980c4be0a8c7aff8fc51a22a6bf8ef643154c2531677bf |
Close
Hashes for python_ucto-0.6.2-cp36-cp36m-manylinux_2_28_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ef148b00d6dc4e0436114828e405933574c610067ab29c839ea2f2e53c540119 |
|
MD5 | d3cc8384d50d053338486da0f3e6988d |
|
BLAKE2b-256 | 3d06e7c53376dc864ead8a62261d3a96c381f17e16598584d48f01caf346a3aa |
Close
Hashes for python_ucto-0.6.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1aa994070063bf05173aa3ac48a797da030542e1bd93ca03ecd1ed3f84d6360c |
|
MD5 | f6d9c0cebe7c1d4f715335319658f03c |
|
BLAKE2b-256 | a92ce76711683a3c0ac86b1506972c824672370f9f9535404ac340982e149145 |
Close
Hashes for python_ucto-0.6.2-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 67f88476226d1f906b319c78588c470f72c8d199d1924e21ede985e71c8c0ee8 |
|
MD5 | 624f653741265be5612e546cec6fca72 |
|
BLAKE2b-256 | cc9256bbdfddc3b59e96f9ae70abf5c4838db3268151a2ebce206220581d9f99 |