This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for proycon from gravatar.com proycon

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: GNU General Public License v3 (GPLv3) (GPLv3)
  • Author: Maarten van Gompel
  • Tags tokenizer , tokenization , tokeniser , tokenisation , nlp , computational_linguistics , ucto
Classifiers
Report project as malware

Project description

The author of this package has not provided a project description

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for proycon from gravatar.com proycon

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: GNU General Public License v3 (GPLv3) (GPLv3)
  • Author: Maarten van Gompel
  • Tags tokenizer , tokenization , tokeniser , tokenisation , nlp , computational_linguistics , ucto
Classifiers

Release history Release notifications | RSS feed

0.6.10

0.6.9

0.6.8

0.6.7

0.6.6

0.6.5

0.6.4

0.6.3

0.6.2

This version

0.6.1

0.6.0

0.5.3

0.5.2

0.5.1

0.5.0

0.4.7

0.4.5

0.4.4

0.4.3

0.4.2

0.3.0

0.2.4

0.2.2

0.2.1

0.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

python-ucto-0.6.1.tar.gz (78.8 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

python_ucto-0.6.1-cp311-cp311-musllinux_1_1_x86_64.whl (26.4 MB view details)

Uploaded CPython 3.11musllinux: musl 1.1+ x86-64

python_ucto-0.6.1-cp311-cp311-manylinux_2_28_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

python_ucto-0.6.1-cp311-cp311-macosx_11_0_arm64.whl (51.8 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

python_ucto-0.6.1-cp311-cp311-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

python_ucto-0.6.1-cp310-cp310-musllinux_1_1_x86_64.whl (26.3 MB view details)

Uploaded CPython 3.10musllinux: musl 1.1+ x86-64

python_ucto-0.6.1-cp310-cp310-manylinux_2_28_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

python_ucto-0.6.1-cp310-cp310-macosx_11_0_arm64.whl (52.4 kB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

python_ucto-0.6.1-cp310-cp310-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

python_ucto-0.6.1-cp39-cp39-musllinux_1_1_x86_64.whl (26.4 MB view details)

Uploaded CPython 3.9musllinux: musl 1.1+ x86-64

python_ucto-0.6.1-cp39-cp39-manylinux_2_28_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.28+ x86-64

python_ucto-0.6.1-cp39-cp39-macosx_11_0_arm64.whl (52.9 kB view details)

Uploaded CPython 3.9macOS 11.0+ ARM64

python_ucto-0.6.1-cp39-cp39-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.9macOS 10.9+ x86-64

python_ucto-0.6.1-cp38-cp38-musllinux_1_1_x86_64.whl (26.4 MB view details)

Uploaded CPython 3.8musllinux: musl 1.1+ x86-64

python_ucto-0.6.1-cp38-cp38-manylinux_2_28_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.8manylinux: glibc 2.28+ x86-64

python_ucto-0.6.1-cp38-cp38-macosx_11_0_arm64.whl (52.0 kB view details)

Uploaded CPython 3.8macOS 11.0+ ARM64

python_ucto-0.6.1-cp38-cp38-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.8macOS 10.9+ x86-64

python_ucto-0.6.1-cp37-cp37m-musllinux_1_1_x86_64.whl (26.3 MB view details)

Uploaded CPython 3.7mmusllinux: musl 1.1+ x86-64

python_ucto-0.6.1-cp37-cp37m-manylinux_2_28_x86_64.whl (25.8 MB view details)

Uploaded CPython 3.7mmanylinux: glibc 2.28+ x86-64

python_ucto-0.6.1-cp37-cp37m-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.7mmacOS 10.9+ x86-64

python_ucto-0.6.1-cp36-cp36m-musllinux_1_1_x86_64.whl (26.3 MB view details)

Uploaded CPython 3.6mmusllinux: musl 1.1+ x86-64

python_ucto-0.6.1-cp36-cp36m-manylinux_2_28_x86_64.whl (25.8 MB view details)

Uploaded CPython 3.6mmanylinux: glibc 2.28+ x86-64

python_ucto-0.6.1-cp36-cp36m-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.6mmacOS 10.9+ x86-64

File details

Details for the file python-ucto-0.6.1.tar.gz.

File metadata

  • Download URL: python-ucto-0.6.1.tar.gz
  • Upload date:
  • Size: 78.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.10.8

File hashes

Hashes for python-ucto-0.6.1.tar.gz
Algorithm Hash digest
SHA256 bd7b01b529ec76ec64ee014f79e2c3af36fe220b3df8ea062b01c3c4d49196a9
MD5 426ecccd7b35a8a5d320852ba6debd06
BLAKE2b-256 9daa8af6e7ddc4cef74c2437ba25db1c31f0542833fb352aa2ad3c0dcb5d1432

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 bfdfa2ab75455465b3966f800642cb18ccd48360701556bf6f2e212642239143
MD5 a981becc57c279993216d4998c76de00
BLAKE2b-256 46fe8aba6cd2c784d5c8e9ad29ccbe8cfa3734ba6e442aaffdf501ebe2e79bf5

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 4ac34c266a57862a78cf37f385fabd6dcd422ad75b9c25896ff971d5b1321edd
MD5 64c635b9952ca99da274bbc76099da90
BLAKE2b-256 f535a3643b5b62b969191d7f6688f6a5b0dfe1d1364b65fe5673851c2a75a7a6

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 3f487ac5f047253ce87566eef4f89842285a13bbdaa9445a8ad1b717e28478c9
MD5 8c127c9dfc7d1457a64e33149068215f
BLAKE2b-256 c8ac3ec587ae4f285c641d2abc0fd1962b289cd370903193d794614ecd8843b9

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 7874af7a6f58c4f2e944f99229967055ad7f868e043c3c465c16e8fd1a65c219
MD5 e731636891a0278261340fbe57504a97
BLAKE2b-256 014cb46b59d41c697c9335f220b147949e641c6eb7aa0e2327e8fa605fba26b2

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 cb61c650a9f8bac89b62b6975ecd11e7892bc188c6f762f8bc8ffaf8ab00142c
MD5 765fa8579d188e5f95b2ad8084c70538
BLAKE2b-256 8c2d76439a0d2a06ec94115532cad1d249ae5fde98dea7741e7dea9bcf53e814

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 e6c0e1150b2c876d3b597c7b62d194220b3560cb5ee7c97ee2d8a630c345a668
MD5 e068e131d19967d028ebab6786aba4b8
BLAKE2b-256 70ca34d468d7156743cb93b5554aaafad6b72ba989c76afc466b7fb100be5737

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 71b25c964f87b49f4e775d09f77f43446303bac3bc9b34b3fc4146d3f4d2848d
MD5 594aeca83c9c7604c48c80719c29a999
BLAKE2b-256 176dbaac482ae79393ac72708a1ac73445ebc956dec555925d511bd41012bfe1

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 567b652b85359c8415def374b316589265a37b4fc3e9b919f81cd2fabe83a6fd
MD5 f6d9c084503da7fa8102ac58af834018
BLAKE2b-256 81f0ed4480820515386f88452e4442ac8f95d23107dc395ce1265e75973a8bd3

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 a8fc634a82d914611e3a7f2c7e7923fece4e07e8c91af33512660958b0828d82
MD5 e36fcd32e2e1b740fd6150960c4b2dda
BLAKE2b-256 6c0e2f5f834c0431defa07c516bde4a32729978da587042b1adf6b1a1038f1ed

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp39-cp39-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp39-cp39-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 edc747b14350429c4e537f6f60f400f1d5eea82c7379ad1f7f26da6bdf199e96
MD5 b3634bbe2280755d847a8d8aa9722ca5
BLAKE2b-256 7f6301c1debbbc6e805ef87e962e4276ee1d00aedf7188489234c6d959776368

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 1732d671db006ebfd2aeb206cc6c6359d91d7272d142bd14a1ad33b18cbaa101
MD5 56a36e83f91dd8ad03dde21434e00e7c
BLAKE2b-256 4cbe4d3b5402ac7910027cf62d8df600b33bcef2cbf6b60aab6f23f741e6200f

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 d8829f1ff6c8579b20c0a568fac3cf5ebf0242796fb02f31ea8ceab4cd97d1ce
MD5 e4ae2f1fa1bcf57c57634ecdac8727ed
BLAKE2b-256 0e1f1a4c8a33beb468b62232f3acf43e2b5cd3a7d469fe3435a229a4c593e7c7

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 51a962f062357f78428ce694660513795d7604d1d63f7a0c2f57c9979ac80aa0
MD5 0455f895bcac43b1c24846518ef9835f
BLAKE2b-256 871c3e375923de0f3d3e74d42094e221f911f944389124a947a37df04ec023c7

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp38-cp38-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp38-cp38-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 4a5b18872781d7b9611590b1237a0016bca7fda050c6784c9e8dbedde7432180
MD5 f136bb31aa6f3546622b3f109d7948d4
BLAKE2b-256 d1f5a0462b4af132f0c74baa9b294f731fd2d4cc6b35b0fe423374bb508c7df2

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 12b9ca9194b6e9983bbae91302f5189bb15d3a889e86bf24f94597b0b1da0f1a
MD5 5ab0bc528fcc01061b366f608b7692ea
BLAKE2b-256 1b58ba4e3a0384a3b8f4cf33a5455baabd0bde89e0faaa20cedef77a5ddd38ac

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 4fb4051c8da95e90bd66d923fdbe3da6ef896608201695dbe33e9926e33c686b
MD5 8885e1d89c0f4937d76c7e832b470a64
BLAKE2b-256 0b17a4bccd37bf1bced0d3142483a509f99fa09cbb7ea25e582c9a7f962064fc

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 6d3d42b702c851bc0cc354c790545de9cf93612265e56d6ec26dfba3bc2b3a2e
MD5 3810dfd96f7647a4d3668a46f543d879
BLAKE2b-256 6d938dc25a6df600ebe9348ac7e931208833f30190458ea5d0917876b7f3f8da

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp37-cp37m-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp37-cp37m-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 5c8e39c7e894f698ddf705e9d38c82ba90e0514ad2d3bc6d4bba02a4f48e9322
MD5 a6c01ff2da487ee60153424eac02f600
BLAKE2b-256 990aeea63398ae238dfb02203f8da89eb8e5807271a7aedc5f9b154acd10df71

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 c9dc49103f68c38bf800509c18790bd332c2ff7b81f2727587c8deed0ca7803e
MD5 40991cd59596ac97e8124dab62b04d85
BLAKE2b-256 a885ee85ccb6add94fd90282396e9fde2d675f323296ebe06190d3692d34bbe2

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp36-cp36m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp36-cp36m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 f5cfcef561f1311da0c7f858ab2e5197be3c923fb96f005e2a27ba89421d19d8
MD5 39256c8118a3db4e6633ea5d5299f63f
BLAKE2b-256 275bfa49f125a4d8de5058c7225730c2b15d44e60e6260c8c9869851e1642387

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp36-cp36m-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp36-cp36m-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 f6045873618a516b0011cf95acfaa176fa9ca7c7b31c004aa3b84ef88d68bc80
MD5 aa7a38e6d651722ed314f469788fcea4
BLAKE2b-256 f9b4e7c80667ff6d70e1cc931e18bb79268f8696380bcaa3207c1fd0ada00561

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.1-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.1-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 df302709623acd46457da7a8d889844a0a13a17135f4b17330619ce2ccd557f4
MD5 a78d7fee08bc7623d32e376cb4504173
BLAKE2b-256 91d08a160fb805a9848c4f9e764f266bfc5297396a6af6e65b9307611166e1e8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page