This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).
Project description
The author of this package has not provided a project description
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
python-ucto-0.6.4.tar.gz
(79.0 kB
view hashes)
Built Distributions
Close
Hashes for python_ucto-0.6.4-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 799e0c37dcf1557303b12d704932d54ad6658e71ea73266b78c072b5c3b86946 |
|
MD5 | 26e4cc3d084f9ac218bdf4747beebdbe |
|
BLAKE2b-256 | aa6e027b44b4a000b68d399810e86f55aba38d54dec7549a7952c6a867297b49 |
Close
Hashes for python_ucto-0.6.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 560e58beffbb34801d45bc4c050910675ecd94e63d72c28799e060ce90014e95 |
|
MD5 | 7a1fead2447078593dbb87b327dc5a63 |
|
BLAKE2b-256 | 8112070f5ebf20a23fda62e243facbd6e22e3cbde06218d77ae5d2df35477567 |
Close
Hashes for python_ucto-0.6.4-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e06ea6d1138de7091bdcc42e2755fefd8de4390f78f9e96664cc6de104fbb76f |
|
MD5 | 35765a7decf15468ed544d5442306b53 |
|
BLAKE2b-256 | c516b67cb4dbd91ed4e48e2b54fc855b2c6b99a6bf97426c37c866d0715dd9e9 |
Close
Hashes for python_ucto-0.6.4-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7c10f6f5ede39d5db6cc0f9c91a65439bba06147cf1911d1b98e253a39aa0e06 |
|
MD5 | dc8a0ca0e5472a2733bf9ea20ead7831 |
|
BLAKE2b-256 | b4036a865579dc78e4edc8dacaebc2513acb253862794df7903d557d3b41df00 |
Close
Hashes for python_ucto-0.6.4-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6b935d878ae4ccac72b220ed36e3503f9f06151ac8d61a63f6b6185b3c66b5e9 |
|
MD5 | 8f7b93b6c56f1e3e7bfea19673acd3ee |
|
BLAKE2b-256 | a406863641ba2f8e01e9422602fbc89bc9678734def98e87297a09e9e8cecd9f |
Close
Hashes for python_ucto-0.6.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 03b98a36a94c132e696bbef750fedbd2d0282bbeb743817fca85a57ec9d93e4b |
|
MD5 | 436b45f6de7a9ac70f2bd7130000b0b0 |
|
BLAKE2b-256 | bbee3290434389527ea683e8770059b2c45f8385dc3326d6a165c4e2be743310 |
Close
Hashes for python_ucto-0.6.4-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 33d6f1c4d00a9f803b04b4b7d0e115ccb6f61d87e068b4190b1a996f0553e307 |
|
MD5 | 9b84885214a41d9434ffe6b250ad2f28 |
|
BLAKE2b-256 | 2d8ba24406965aadb475492935732713d5789c45c7558b84ef5ee3fc24cb2f69 |
Close
Hashes for python_ucto-0.6.4-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 64b7ff62646414717936c18d3deb749b482cf56d687f9bcd000c139c4f95a02e |
|
MD5 | 3d579c3f87dca90c618e1c0eb7865348 |
|
BLAKE2b-256 | 4a84a346853b6537c1aefb0d26d8cd7b2213a65e232ab2d5a9c369f245aba0f6 |
Close
Hashes for python_ucto-0.6.4-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec8bc9c64901994cfe5cb081f08d218378102b0272838f7888eb2b7399bbeb89 |
|
MD5 | 84d4bfdb792a9f32483fa407da90335f |
|
BLAKE2b-256 | edff9e83d2c1741d58ff99b0bc8e592ccec6e8ad2a8139b9e7f0144c9a44f967 |
Close
Hashes for python_ucto-0.6.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | def016178bb139dbad337675c2b3fb477393e608831a46f5e8aa4f7d105adb6b |
|
MD5 | 0f138afaf06cd6b4eda238d62c3d3adb |
|
BLAKE2b-256 | 8e302b28d2c4ae51ea9cba2a47a81c7cf3a01d21e790ac7e635d2803349fbfe6 |
Close
Hashes for python_ucto-0.6.4-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a19b88f5deefcb13717f800c09ec8c754230d8f14c90d6dad7bdbaa15cee38ae |
|
MD5 | ce10ec989134a49d71b9c756dd0064fb |
|
BLAKE2b-256 | 5621cb8419858ab0913730c62694099b4bbdcaa18c38f5ef1937118ea6b48c36 |
Close
Hashes for python_ucto-0.6.4-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6dbd9cde4017c01cfcc2cd51385e4b7b58a00157154665e5fce1614ca489bba8 |
|
MD5 | 193da162a3ac9dedf8c93bb45ab238a9 |
|
BLAKE2b-256 | 4e0e787e30bdecc0b3aca626a2d4be80a51784d31d66bd70cca5753b50aa5909 |
Close
Hashes for python_ucto-0.6.4-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6294e96eb8b98393406fb72eae2c8dd684ac5f4a60995e6f1232475ba23cf593 |
|
MD5 | b2e4b54af80b20effe330f1cc64fdd93 |
|
BLAKE2b-256 | 322d3f4f564d4d0ba3c6557aff81702afd054c207b8bcf4739ae89325f891400 |
Close
Hashes for python_ucto-0.6.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2694f02a9add8b0c6e891acf815e2ab758418637240788803ef340d53fa6f748 |
|
MD5 | 4de6dc6977ccbb98eef224934c44cb19 |
|
BLAKE2b-256 | d3dc6ae044d4d646a8b81865af1992e518aca22b394c206b5c4217a14b236cb1 |
Close
Hashes for python_ucto-0.6.4-cp38-cp38-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 586ef5a5555ce2a2042b601bc3cc618bc6b618cc4e59ad5a710cf825f3f03c54 |
|
MD5 | 5a81648ca32dab81eca47d978209d4e7 |
|
BLAKE2b-256 | 3103ec99b61a3250fa050c8536f84bc82505fbea1155b969c1ef4021c459dfef |
Close
Hashes for python_ucto-0.6.4-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 66dc5ed998c742e2bc5a3dbe72419a6efde49083fc8bc4d004aed68dbf936ab9 |
|
MD5 | 773b1f8ddb9139fe4099849523948c79 |
|
BLAKE2b-256 | 77f2264ae2e3f7575423ca3c1e09dc231e5a3528f61b48be90a6207d9a2bba59 |
Close
Hashes for python_ucto-0.6.4-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 059e2b252e946784f8458d12cc972f2175161420c4fe9141c95983d2bebd31b7 |
|
MD5 | cf3babee354d50a32dbc0db1c43bce3d |
|
BLAKE2b-256 | 653e645a906dc6ea7b81c11da0a2accbfb6f62a7106963ce15aebf36866dde3a |
Close
Hashes for python_ucto-0.6.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0acd42092dbf586e055e07ca7457ece92aec2440dbd58e5eab47b8696b0d8277 |
|
MD5 | b7f9e7c92f9f4f1c6a74a0667624f7c0 |
|
BLAKE2b-256 | b974d7f649d58af0f2b3b9d028d3b59ca01bd04ded39207da3e57e061b08cef8 |
Close
Hashes for python_ucto-0.6.4-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 172fd2c36060a7beceda54242d3c00ad17fa0778ccb8a57e64a37711ae679be0 |
|
MD5 | ff01825ee0eb119c02b48b25e479d12f |
|
BLAKE2b-256 | 5f361c114bfe0c79aae0d898e439f5685d702185f45d0e644a8b718bc70d87b4 |
Close
Hashes for python_ucto-0.6.4-cp36-cp36m-musllinux_1_1_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0fe63ff7a4ce2debe8990f1d87a28c80435ad925c8d6f7eb3d8768cb6a3f14ae |
|
MD5 | 4898847030922afbed4f0757ffb1b78f |
|
BLAKE2b-256 | adf26af6a998d2c4dc6c118fa73524be64d8fa9f91ed42c60167fe9aea206546 |
Close
Hashes for python_ucto-0.6.4-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 39e9835f4def9407df88d09b07ba9f5db07309c8f2dd0b14a04d0dd408c8e947 |
|
MD5 | b68074f7275e0f76490af39a82d0d343 |
|
BLAKE2b-256 | 1aa67fe9076bad46e2b2ff57d82d8486bb04e79f9ffddfdad097391459b26c52 |
Close
Hashes for python_ucto-0.6.4-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 09552e269a870a42e634895b1a78ba1bd072e7b50ca0dd69bc30603766f1ad0a |
|
MD5 | fd36b853d501af82e394ca6c80e77d3f |
|
BLAKE2b-256 | c0917d73bb5083756b8a661a13f53d45ddcfdd9c6560cae42aee2541ab546369 |