This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for proycon from gravatar.com proycon

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: GNU General Public License v3 (GPLv3) (GPLv3)
  • Author: Maarten van Gompel
  • Tags tokenizer , tokenization , tokeniser , tokenisation , nlp , computational_linguistics , ucto
Classifiers
Report project as malware

Project description

The author of this package has not provided a project description

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for proycon from gravatar.com proycon

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: GNU General Public License v3 (GPLv3) (GPLv3)
  • Author: Maarten van Gompel
  • Tags tokenizer , tokenization , tokeniser , tokenisation , nlp , computational_linguistics , ucto
Classifiers

Release history Release notifications | RSS feed

0.6.10

0.6.9

0.6.8

0.6.7

This version

0.6.6

0.6.5

0.6.4

0.6.3

0.6.2

0.6.1

0.6.0

0.5.3

0.5.2

0.5.1

0.5.0

0.4.7

0.4.5

0.4.4

0.4.3

0.4.2

0.3.0

0.2.4

0.2.2

0.2.1

0.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

python-ucto-0.6.6.tar.gz (103.6 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

python_ucto-0.6.6-cp312-cp312-musllinux_1_1_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.12musllinux: musl 1.1+ x86-64

python_ucto-0.6.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.2 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

python_ucto-0.6.6-cp312-cp312-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.12macOS 10.9+ x86-64

python_ucto-0.6.6-cp311-cp311-musllinux_1_1_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.11musllinux: musl 1.1+ x86-64

python_ucto-0.6.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.2 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

python_ucto-0.6.6-cp311-cp311-macosx_11_0_arm64.whl (15.4 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

python_ucto-0.6.6-cp311-cp311-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

python_ucto-0.6.6-cp310-cp310-musllinux_1_1_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.10musllinux: musl 1.1+ x86-64

python_ucto-0.6.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.2 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

python_ucto-0.6.6-cp310-cp310-macosx_11_0_arm64.whl (15.4 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

python_ucto-0.6.6-cp310-cp310-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

python_ucto-0.6.6-cp39-cp39-musllinux_1_1_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.9musllinux: musl 1.1+ x86-64

python_ucto-0.6.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.2 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

python_ucto-0.6.6-cp39-cp39-macosx_11_0_arm64.whl (15.4 MB view details)

Uploaded CPython 3.9macOS 11.0+ ARM64

python_ucto-0.6.6-cp39-cp39-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.9macOS 10.9+ x86-64

python_ucto-0.6.6-cp38-cp38-musllinux_1_1_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.8musllinux: musl 1.1+ x86-64

python_ucto-0.6.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.2 MB view details)

Uploaded CPython 3.8manylinux: glibc 2.17+ x86-64

python_ucto-0.6.6-cp38-cp38-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.8macOS 10.9+ x86-64

python_ucto-0.6.6-cp37-cp37m-musllinux_1_1_x86_64.whl (25.8 MB view details)

Uploaded CPython 3.7mmusllinux: musl 1.1+ x86-64

python_ucto-0.6.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.2 MB view details)

Uploaded CPython 3.7mmanylinux: glibc 2.17+ x86-64

python_ucto-0.6.6-cp37-cp37m-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.7mmacOS 10.9+ x86-64

python_ucto-0.6.6-cp36-cp36m-musllinux_1_1_x86_64.whl (25.8 MB view details)

Uploaded CPython 3.6mmusllinux: musl 1.1+ x86-64

python_ucto-0.6.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.1 MB view details)

Uploaded CPython 3.6mmanylinux: glibc 2.17+ x86-64

python_ucto-0.6.6-cp36-cp36m-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.6mmacOS 10.9+ x86-64

File details

Details for the file python-ucto-0.6.6.tar.gz.

File metadata

  • Download URL: python-ucto-0.6.6.tar.gz
  • Upload date:
  • Size: 103.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for python-ucto-0.6.6.tar.gz
Algorithm Hash digest
SHA256 7019726d9a793e6ac822f1fad9a3ef9bbcd0468a4fd0c26c53d17f59162c95f6
MD5 8cc140dae9a042dd8e356d05535e21b9
BLAKE2b-256 75a27f186b9a586583eed9621395417a3b205ebd772b4dee0d0fb05d0f78ce5e

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp312-cp312-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 a9a5e2e7031a75cda08e5f0cbc7a60edb4dae1ae413958dc6289f58c90c61ec3
MD5 53481a62fbd8a3885d07b1119276ffea
BLAKE2b-256 406ca4bf1b83263ced4dc07db11cf854c4d45b9156ce33546d5a304c5051c228

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8163c1a8552aaa119ab94d64a53faa3f492eab01d57d22c57f503378a6255f4b
MD5 0809a0f5debb72f5d477e331108736ea
BLAKE2b-256 cada45021014496106d08ccc3a496f00b2de636c47bde977dcc835a31278e98e

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp312-cp312-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp312-cp312-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 bceb8b0e8851b6e8cdb46e27d242f3ba785142ee1cc003d75db743dc80d335da
MD5 c6cda81bd93acf81c38a23a8b1f478ae
BLAKE2b-256 c03e425ab310e8c3efe90baeda20f7dd21b993c93cb9e95addb3373846813212

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 dd427a02f399d6dc302376805ab68c5450417566415492cd167983419aa459e0
MD5 4c153a8da4885beb995cad3b1b4bb31c
BLAKE2b-256 808c1f648b989c71c9ff93b6a935072b8e04ecbf676b69b05c3743f76ad67d48

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 9cf2c55fb5a7770880a9a41734491e8616a8b26d4b62a7a3d1cbc2c99c24486b
MD5 4d9c8e4e0bed484bbf58e7ced38ea4d5
BLAKE2b-256 6b45fb6d3bea57f7c442e73ba814f6a2f5cf5111ffa48f190cd8a57fe2832d08

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 39acdba426e46eca7ee31b9807426e51a929cac10313d216a85a37cd65374eaf
MD5 bedd9af9eafb7dad355fbfb0e2d3a8b4
BLAKE2b-256 6c6d61333afeedb5a05baa0a7fdd46242163e20e4f8c450942c4f60a0c56a503

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 bb1c3bdcdf639d38b3988b503c37141ebc5a8c5afb7b1ef130112490c509fe23
MD5 ae0f5639acc9066cf3a349d2f0dc18bc
BLAKE2b-256 e49879ab206c4849fdfe672f44b07fcdfc154aa3f26cd04daeddf3d9595da091

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 31cf3ddc73bdf7d8ee351dfc7417d929d3ce4ad4f41bf75fc4b50ff273ef4c21
MD5 6bfeca74a4589f3dd38e1482d5b3cc4a
BLAKE2b-256 9ffec23594268064d1b61b00e2a96ae9d0f9057809e34ee400b89b158da24e03

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1123bf13a104dcff5905d97448f5b0ebcc489344e39726d5269d943963a62f7b
MD5 3fb808eb50cf64390845ce81c3dd5f59
BLAKE2b-256 906e0c4f8f6aa067b339efce1273741801018c2be9c83650dceb3f7fd05d9a7e

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8026099720cfc8ae0fd3472c16ed0c6ed868a25b0ecbf2ec05a585eefbdddb77
MD5 4502264ad063e11716005a7556707b2f
BLAKE2b-256 1d33842f5e4de1686e5bcda3c2dfe3dabfd424611e0ca22cae9c5e0986cd2a53

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 08fde82b0a81876fdac2935fa60018e7cdbd727d85ad1f115273249ee9b5b9eb
MD5 44aa6f5d353573899e42eeddffea573f
BLAKE2b-256 95dccaf3c2c28496fd5a9d17de190188a0086ae7cc0914ed2166cc3403131114

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 338384180e3dbb01ab0d4aa4ec8ffb289366d07c10264fa4a6f7b54f3cf886a7
MD5 d548fe9325ed421111cc4bf03fcd9925
BLAKE2b-256 dcd80433cdbaeb4b35b81bc6129c643fc2aa7138d633b47810ef7a2f18482bb0

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b2ec2665e8b67e512a12b06b6a55a424ba56f8ba4e3d845a3ce3149fb2fc58cb
MD5 162c35d8ffbd5236bc889d9050b22ae6
BLAKE2b-256 2ed7cdfa013a274f2f8308da7ed0fdcbe0064477b1c904852b2c056de0c8c767

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f3e1d4973cbf4b5c6cfdb7e0eea03e5853e25561df6cae785a8518216036de67
MD5 b9448fdf63d3899744f5474b6df8d2ea
BLAKE2b-256 2828d878c5981f735973cf99702c5b276c79d4b8342f2f8be4507a42ca0b552c

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 cfbed7400b49150c51b56bc59ca0347411e049857ae9daa57a49e65fcf92e7dc
MD5 78170c94cecb511bd756d9b443831344
BLAKE2b-256 cee79d405523f87498094a1c8c364d27d34d4365729744cdd989bbf23d09eaee

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 e61d7f743385fd8884edd674d59d4d4fca8d493f5379f545fe893ee6271116be
MD5 0dc8793c00eb181c0f54b2e64d33e7da
BLAKE2b-256 51c6baefdd43353e13f365b74199e232662b3f59c1fc5a77fb5c35149d14ea43

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b4d78366be0248d396b28c1e85212d6ad15f3cd020325273d19c2deb31058873
MD5 5b3e5f48a63773051979cc4ce054cdaf
BLAKE2b-256 55ec9b1ed6486995e458e9c7dd5dfcbaeb6a3d4a7a7549c741f1225748575cca

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 9f03dd9e460991a9f2b076bddc39d7ee3db36f4febbffeee1d2525506c320e87
MD5 c544aadd9ff1eec62e065c04bc979896
BLAKE2b-256 da2794ff20b115ccc58aaa8e37bb4e382ae2e3cfe9b8e48191bc76e7a1800249

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 458e10682dc192d9185607358fdf1e22fcbf5529176b7eaeb151591c3ce1595c
MD5 a98538583e59db6892d67060276954d2
BLAKE2b-256 f50e381ee7b52979deade0c78b3c387ef53a4a77f3df3a9fd701eba9a05a9e0e

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e0d31e00b37183494755abd90cc7f9ec4b7b333e00a3000871c63bee30ab6d76
MD5 219f8f159b74ee6f08d10c8cf2a6735b
BLAKE2b-256 42e2c733815c1e23742c644ef9355182154c10ef12ed00ce847a0f45db05d5ba

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 38d48fcd97c296c3b00b4ba7f13d5234d5672f66b9c64d44456208db7d081c29
MD5 4c2ddfaeec9d7b072a8ee305ddfaa879
BLAKE2b-256 a073270015af6d849def61853782f2155719a843bb8a2fe4bcacca5524a7397c

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp36-cp36m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp36-cp36m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 dc6e5617e3df413e5889c6ec368d737701cbd3fa10288856225912da3ae9b646
MD5 e16b2c474007b0f8dc1ca9505e7de708
BLAKE2b-256 313219234e9a15f973935075e69fb7f857856e04ec7ffe739c23530d90d59617

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 9733e01b23c1c1fd2d7636ce7019a8ef92a5b183843198499164bc2e1914ea11
MD5 4f578eb524777ec97d412675320f8273
BLAKE2b-256 c082e416a7f9fa3b76ae324770497bd70302e05d672c5419b2cdd266a9706188

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.6-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.6-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 8369a164c216388b169162fcf9032a11deb3594e33791f52415c7f9b08d62128
MD5 3a38026bd5cd9cbafa233d11807471af
BLAKE2b-256 2096987bf0caf7404e18a2b292ef839475bf1eac9d91bf548216e984e01a91dd

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page