This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).
Project description
The author of this package has not provided a project description
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
python-ucto-0.6.6.tar.gz
(103.6 kB
view hashes)
Built Distributions
Close
Hashes for python_ucto-0.6.6-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a9a5e2e7031a75cda08e5f0cbc7a60edb4dae1ae413958dc6289f58c90c61ec3 |
|
MD5 | 53481a62fbd8a3885d07b1119276ffea |
|
BLAKE2b-256 | 406ca4bf1b83263ced4dc07db11cf854c4d45b9156ce33546d5a304c5051c228 |
Close
Hashes for python_ucto-0.6.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8163c1a8552aaa119ab94d64a53faa3f492eab01d57d22c57f503378a6255f4b |
|
MD5 | 0809a0f5debb72f5d477e331108736ea |
|
BLAKE2b-256 | cada45021014496106d08ccc3a496f00b2de636c47bde977dcc835a31278e98e |
Close
Hashes for python_ucto-0.6.6-cp312-cp312-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bceb8b0e8851b6e8cdb46e27d242f3ba785142ee1cc003d75db743dc80d335da |
|
MD5 | c6cda81bd93acf81c38a23a8b1f478ae |
|
BLAKE2b-256 | c03e425ab310e8c3efe90baeda20f7dd21b993c93cb9e95addb3373846813212 |
Close
Hashes for python_ucto-0.6.6-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dd427a02f399d6dc302376805ab68c5450417566415492cd167983419aa459e0 |
|
MD5 | 4c153a8da4885beb995cad3b1b4bb31c |
|
BLAKE2b-256 | 808c1f648b989c71c9ff93b6a935072b8e04ecbf676b69b05c3743f76ad67d48 |
Close
Hashes for python_ucto-0.6.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9cf2c55fb5a7770880a9a41734491e8616a8b26d4b62a7a3d1cbc2c99c24486b |
|
MD5 | 4d9c8e4e0bed484bbf58e7ced38ea4d5 |
|
BLAKE2b-256 | 6b45fb6d3bea57f7c442e73ba814f6a2f5cf5111ffa48f190cd8a57fe2832d08 |
Close
Hashes for python_ucto-0.6.6-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 39acdba426e46eca7ee31b9807426e51a929cac10313d216a85a37cd65374eaf |
|
MD5 | bedd9af9eafb7dad355fbfb0e2d3a8b4 |
|
BLAKE2b-256 | 6c6d61333afeedb5a05baa0a7fdd46242163e20e4f8c450942c4f60a0c56a503 |
Close
Hashes for python_ucto-0.6.6-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bb1c3bdcdf639d38b3988b503c37141ebc5a8c5afb7b1ef130112490c509fe23 |
|
MD5 | ae0f5639acc9066cf3a349d2f0dc18bc |
|
BLAKE2b-256 | e49879ab206c4849fdfe672f44b07fcdfc154aa3f26cd04daeddf3d9595da091 |
Close
Hashes for python_ucto-0.6.6-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 31cf3ddc73bdf7d8ee351dfc7417d929d3ce4ad4f41bf75fc4b50ff273ef4c21 |
|
MD5 | 6bfeca74a4589f3dd38e1482d5b3cc4a |
|
BLAKE2b-256 | 9ffec23594268064d1b61b00e2a96ae9d0f9057809e34ee400b89b158da24e03 |
Close
Hashes for python_ucto-0.6.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1123bf13a104dcff5905d97448f5b0ebcc489344e39726d5269d943963a62f7b |
|
MD5 | 3fb808eb50cf64390845ce81c3dd5f59 |
|
BLAKE2b-256 | 906e0c4f8f6aa067b339efce1273741801018c2be9c83650dceb3f7fd05d9a7e |
Close
Hashes for python_ucto-0.6.6-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8026099720cfc8ae0fd3472c16ed0c6ed868a25b0ecbf2ec05a585eefbdddb77 |
|
MD5 | 4502264ad063e11716005a7556707b2f |
|
BLAKE2b-256 | 1d33842f5e4de1686e5bcda3c2dfe3dabfd424611e0ca22cae9c5e0986cd2a53 |
Close
Hashes for python_ucto-0.6.6-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 08fde82b0a81876fdac2935fa60018e7cdbd727d85ad1f115273249ee9b5b9eb |
|
MD5 | 44aa6f5d353573899e42eeddffea573f |
|
BLAKE2b-256 | 95dccaf3c2c28496fd5a9d17de190188a0086ae7cc0914ed2166cc3403131114 |
Close
Hashes for python_ucto-0.6.6-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 338384180e3dbb01ab0d4aa4ec8ffb289366d07c10264fa4a6f7b54f3cf886a7 |
|
MD5 | d548fe9325ed421111cc4bf03fcd9925 |
|
BLAKE2b-256 | dcd80433cdbaeb4b35b81bc6129c643fc2aa7138d633b47810ef7a2f18482bb0 |
Close
Hashes for python_ucto-0.6.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b2ec2665e8b67e512a12b06b6a55a424ba56f8ba4e3d845a3ce3149fb2fc58cb |
|
MD5 | 162c35d8ffbd5236bc889d9050b22ae6 |
|
BLAKE2b-256 | 2ed7cdfa013a274f2f8308da7ed0fdcbe0064477b1c904852b2c056de0c8c767 |
Close
Hashes for python_ucto-0.6.6-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3e1d4973cbf4b5c6cfdb7e0eea03e5853e25561df6cae785a8518216036de67 |
|
MD5 | b9448fdf63d3899744f5474b6df8d2ea |
|
BLAKE2b-256 | 2828d878c5981f735973cf99702c5b276c79d4b8342f2f8be4507a42ca0b552c |
Close
Hashes for python_ucto-0.6.6-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cfbed7400b49150c51b56bc59ca0347411e049857ae9daa57a49e65fcf92e7dc |
|
MD5 | 78170c94cecb511bd756d9b443831344 |
|
BLAKE2b-256 | cee79d405523f87498094a1c8c364d27d34d4365729744cdd989bbf23d09eaee |
Close
Hashes for python_ucto-0.6.6-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e61d7f743385fd8884edd674d59d4d4fca8d493f5379f545fe893ee6271116be |
|
MD5 | 0dc8793c00eb181c0f54b2e64d33e7da |
|
BLAKE2b-256 | 51c6baefdd43353e13f365b74199e232662b3f59c1fc5a77fb5c35149d14ea43 |
Close
Hashes for python_ucto-0.6.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b4d78366be0248d396b28c1e85212d6ad15f3cd020325273d19c2deb31058873 |
|
MD5 | 5b3e5f48a63773051979cc4ce054cdaf |
|
BLAKE2b-256 | 55ec9b1ed6486995e458e9c7dd5dfcbaeb6a3d4a7a7549c741f1225748575cca |
Close
Hashes for python_ucto-0.6.6-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9f03dd9e460991a9f2b076bddc39d7ee3db36f4febbffeee1d2525506c320e87 |
|
MD5 | c544aadd9ff1eec62e065c04bc979896 |
|
BLAKE2b-256 | da2794ff20b115ccc58aaa8e37bb4e382ae2e3cfe9b8e48191bc76e7a1800249 |
Close
Hashes for python_ucto-0.6.6-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 458e10682dc192d9185607358fdf1e22fcbf5529176b7eaeb151591c3ce1595c |
|
MD5 | a98538583e59db6892d67060276954d2 |
|
BLAKE2b-256 | f50e381ee7b52979deade0c78b3c387ef53a4a77f3df3a9fd701eba9a05a9e0e |
Close
Hashes for python_ucto-0.6.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e0d31e00b37183494755abd90cc7f9ec4b7b333e00a3000871c63bee30ab6d76 |
|
MD5 | 219f8f159b74ee6f08d10c8cf2a6735b |
|
BLAKE2b-256 | 42e2c733815c1e23742c644ef9355182154c10ef12ed00ce847a0f45db05d5ba |
Close
Hashes for python_ucto-0.6.6-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 38d48fcd97c296c3b00b4ba7f13d5234d5672f66b9c64d44456208db7d081c29 |
|
MD5 | 4c2ddfaeec9d7b072a8ee305ddfaa879 |
|
BLAKE2b-256 | a073270015af6d849def61853782f2155719a843bb8a2fe4bcacca5524a7397c |
Close
Hashes for python_ucto-0.6.6-cp36-cp36m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dc6e5617e3df413e5889c6ec368d737701cbd3fa10288856225912da3ae9b646 |
|
MD5 | e16b2c474007b0f8dc1ca9505e7de708 |
|
BLAKE2b-256 | 313219234e9a15f973935075e69fb7f857856e04ec7ffe739c23530d90d59617 |
Close
Hashes for python_ucto-0.6.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9733e01b23c1c1fd2d7636ce7019a8ef92a5b183843198499164bc2e1914ea11 |
|
MD5 | 4f578eb524777ec97d412675320f8273 |
|
BLAKE2b-256 | c082e416a7f9fa3b76ae324770497bd70302e05d672c5419b2cdd266a9706188 |
Close
Hashes for python_ucto-0.6.6-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8369a164c216388b169162fcf9032a11deb3594e33791f52415c7f9b08d62128 |
|
MD5 | 3a38026bd5cd9cbafa233d11807471af |
|
BLAKE2b-256 | 2096987bf0caf7404e18a2b292ef839475bf1eac9d91bf548216e984e01a91dd |