This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).
Project description
The author of this package has not provided a project description
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
python-ucto-0.6.3.tar.gz
(79.0 kB
view hashes)
Built Distributions
Close
Hashes for python_ucto-0.6.3-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3bdb4546f9c1a94a7a288053cf4c88df0417f8b054555ca04c921f5abf8138f |
|
MD5 | 8e6b9a9c355f2c09b50aca43cb407f07 |
|
BLAKE2b-256 | 40f5cd515f94c2059400bdf71d2fd5a109d3e2a6bdd8c204059f6803815d2ace |
Close
Hashes for python_ucto-0.6.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | adf01cffbcffac15ac9ac265fbb1c826d920c9e20214179646cbd8349daf3ba1 |
|
MD5 | 77995dc4a4173a4cffc3ee368ce150b9 |
|
BLAKE2b-256 | 985d5bfa13ee8a29f6e8ecfaccbcac408ab4259f1e0d95dc4e7b72c5d93b7f77 |
Close
Hashes for python_ucto-0.6.3-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9c859686487915323653ccecbb78c5c911bd906c2c6efba0e8f91ba02f3ad653 |
|
MD5 | c0ae138a6376724537ea8a6c9445ca50 |
|
BLAKE2b-256 | d1da950a08291dfbd29dac18520d61e22ff9d0a677111fe7965f0e340fc3b057 |
Close
Hashes for python_ucto-0.6.3-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 360495dd270694c3b8d89437bb7bad782ce0dbaaf455696831535ac4aba87525 |
|
MD5 | df0aa417488dfa2cc64f0b13ea2424a2 |
|
BLAKE2b-256 | d2440dc1966866d45724d0fddb94a87f63ee5b0ed184e03273ba131cdc2797dc |
Close
Hashes for python_ucto-0.6.3-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ef691975dc3b73d16823481c5c0e12300627d4e7de061d52b94a218b30856769 |
|
MD5 | 88c1791c876b75fe72e27bc1d2eea541 |
|
BLAKE2b-256 | 2e36de8d646709a48527ea1874ebc58032211eeabfa44bbd8584744f4e0f0f5f |
Close
Hashes for python_ucto-0.6.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1f37e7aeca610897e1a4dc8471c10c177de225a269b282e8e7e3a303637a4c49 |
|
MD5 | 86f12ddb2753b2312fed6b834f0ac0dd |
|
BLAKE2b-256 | 9189b2af40de060fef1b6a5b74aece0fefb2c519f7eb73b0fd376d5277584f27 |
Close
Hashes for python_ucto-0.6.3-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 540f6b1d7b785d5f5134a8584c5ce251a0d9f64e796bebf2756edc4b7aa8ee32 |
|
MD5 | 23ceb5148faa54e513ccf4c398f644eb |
|
BLAKE2b-256 | 533ca904b1de3005856f96c133c56613f7b263cd7ed90d9fb659e1e1f66324bf |
Close
Hashes for python_ucto-0.6.3-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 39d3fa148a58f4e94594e561ee103ebd8a04ccca88c15f044118dbfa9dedd0d9 |
|
MD5 | f5c7f1d917c5493fbf2ae69a043feeb8 |
|
BLAKE2b-256 | 8fa45d870a03f4071ed1d477a32e3c856d7e66f3d9fa74118e25520f6850536f |
Close
Hashes for python_ucto-0.6.3-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 07eebbb6849e1d85a0e6915e4ca103e535a5a58d38760893f17f88f80df2af41 |
|
MD5 | f06cec978de33d0e4048eb78303a9531 |
|
BLAKE2b-256 | 5b9f085e62e6798b934b2928e47f8ff5092d1f7c3047078426aa3f58e838b92f |
Close
Hashes for python_ucto-0.6.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3b2ac8a9c0bc71f9f6c9aa30f148169ee48f40cba459d12c4b983f7d6849e923 |
|
MD5 | ce5a280403d1990f0a9e7b8513890b92 |
|
BLAKE2b-256 | eac01c33557f9647e7b09e1639d2b060346dc8bffc6461bd1ee770ee7dfebafb |
Close
Hashes for python_ucto-0.6.3-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4df0d8f3fca29a3305e3c6f2dcdfe99d860810f03bcd37ecd49bcb2d237792a7 |
|
MD5 | f011e056d8b5f7e6711b43e41a117ea7 |
|
BLAKE2b-256 | 7459afe6f4c0ba490872a9ab28463c0e1f3769ccc94c17c3803d4877c89c0411 |
Close
Hashes for python_ucto-0.6.3-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d5299e83ad06277758a189752e6af20c29e5da05e6a28da97b9a560f8a443f59 |
|
MD5 | f4711494f4b5aff2b466939c6f0c5014 |
|
BLAKE2b-256 | 86a1082e1815f0a8e8ea106cd57e698ed1f4ed6a1b045b44c4f0129edc183e79 |
Close
Hashes for python_ucto-0.6.3-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 64429aed0e816dc4e1f04488b9cfb5d3b743616c68e10a400fec85480b3668a0 |
|
MD5 | 7a404f5b1f4557beb2049fae0c669063 |
|
BLAKE2b-256 | d8fbb057d83b83e4c6591253d820daac4b202d05e400deeeaecb8e79bc4aea9b |
Close
Hashes for python_ucto-0.6.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d0e67ac85d1c95f85deddf1698e23ac4ed022efd0c77f617b100b7f71afbf73c |
|
MD5 | ef6ae5e239dcdf6f695672d3e19ab7ed |
|
BLAKE2b-256 | 02683d1c5b397be76ee21aa0e1471aac79156b835ab8fd3036e3d127fbcad193 |
Close
Hashes for python_ucto-0.6.3-cp38-cp38-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e40cbbfabc9cb4ad5eb034c636eb70993be69a63dab3889b509852175656f3c0 |
|
MD5 | 5ea5922209a9caf22a4a7c3e441bb186 |
|
BLAKE2b-256 | d0d42245e35036b85e7a4a92b6105eb70362a5fb28976b2f7691142a88836a1e |
Close
Hashes for python_ucto-0.6.3-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8fac5bf9ce61e504a71ba9cabe1e6c66ab24d229d1a978bd16d57a55ea0f88f |
|
MD5 | 733721171f065d5e7bc812a7dc794a89 |
|
BLAKE2b-256 | 1733e86679ed1d1931463ba228697ad7b5bc9b6828e3ec288c0fad5385ce1525 |
Close
Hashes for python_ucto-0.6.3-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 610a043729ed3f68a17a8626300adcdf63225926fcc66764bd0d44267af8591e |
|
MD5 | 0bf30da445ce626ab80bda3d265b34a2 |
|
BLAKE2b-256 | 7985e5c594b931d053f531395df3f39376fa826b3d000f5d3356cae89eeffbe1 |
Close
Hashes for python_ucto-0.6.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5f92d17ecf4d015d2cdca48bc630b3ecba6ca61172354ec96e45d5d57dd2ddc7 |
|
MD5 | 29da5c1773841524b2ee39baa3d140fd |
|
BLAKE2b-256 | fa942bcc221bbfc87940eb9d2dd9084348e73e23d4fb832b94b6b8b5605e104d |
Close
Hashes for python_ucto-0.6.3-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0d5c73fd27e3a553b2c22b829ca00a50821113670a2d03939a584362e87bab9f |
|
MD5 | 649960c06b472a245cfae4e0bc6f1f8f |
|
BLAKE2b-256 | 92e01a0cac1c927655d0a52206ac57753ae7b78934ecd4b2f9d6d1a2b42469e1 |
Close
Hashes for python_ucto-0.6.3-cp36-cp36m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 30521eec33986aec885425b867a74f15205d7cb45a3d08085307fd494ca3bdbd |
|
MD5 | 9ec100aefe41f02af0ad5304b0ea3e47 |
|
BLAKE2b-256 | 0543e9352473d9550987f1329cdebdeb4f5cc9ced0892d4d49d2f27a7795c7a0 |
Close
Hashes for python_ucto-0.6.3-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3d15f7d237cc92ee951d33aa7e2d998c291e98a93ea59879716586498fa93bca |
|
MD5 | 62bdec71aa72827abc7a616335d681c5 |
|
BLAKE2b-256 | 420bd9e00f720fa114399feee37f03e0496416c6451e6d2c7c92ad7b3b2a3437 |
Close
Hashes for python_ucto-0.6.3-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a37d0d8b84f70017c2ffc7b6b08ca6106443832bf8467598e331cb381d767c84 |
|
MD5 | fc098172f792bf5be54c6a25b87e87df |
|
BLAKE2b-256 | 32e9de55b4b6cd20519b325b3ae104be9f783870d5bc278e90d972eb069410b5 |