This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for proycon from gravatar.com proycon

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: GNU General Public License v3 (GPLv3) (GPLv3)
  • Author: Maarten van Gompel
  • Tags tokenizer , tokenization , tokeniser , tokenisation , nlp , computational_linguistics , ucto
Classifiers
Report project as malware

Project description

The author of this package has not provided a project description

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for proycon from gravatar.com proycon

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: GNU General Public License v3 (GPLv3) (GPLv3)
  • Author: Maarten van Gompel
  • Tags tokenizer , tokenization , tokeniser , tokenisation , nlp , computational_linguistics , ucto
Classifiers

Release history Release notifications | RSS feed

0.6.10

0.6.9

0.6.8

This version

0.6.7

0.6.6

0.6.5

0.6.4

0.6.3

0.6.2

0.6.1

0.6.0

0.5.3

0.5.2

0.5.1

0.5.0

0.4.7

0.4.5

0.4.4

0.4.3

0.4.2

0.3.0

0.2.4

0.2.2

0.2.1

0.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

python-ucto-0.6.7.tar.gz (104.7 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

python_ucto-0.6.7-cp312-cp312-musllinux_1_1_x86_64.whl (26.0 MB view details)

Uploaded CPython 3.12musllinux: musl 1.1+ x86-64

python_ucto-0.6.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

python_ucto-0.6.7-cp312-cp312-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.12macOS 10.9+ x86-64

python_ucto-0.6.7-cp311-cp311-musllinux_1_1_x86_64.whl (26.0 MB view details)

Uploaded CPython 3.11musllinux: musl 1.1+ x86-64

python_ucto-0.6.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

python_ucto-0.6.7-cp311-cp311-macosx_11_0_arm64.whl (15.4 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

python_ucto-0.6.7-cp311-cp311-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

python_ucto-0.6.7-cp310-cp310-musllinux_1_1_x86_64.whl (26.0 MB view details)

Uploaded CPython 3.10musllinux: musl 1.1+ x86-64

python_ucto-0.6.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

python_ucto-0.6.7-cp310-cp310-macosx_11_0_arm64.whl (15.4 MB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

python_ucto-0.6.7-cp310-cp310-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

python_ucto-0.6.7-cp39-cp39-musllinux_1_1_x86_64.whl (26.0 MB view details)

Uploaded CPython 3.9musllinux: musl 1.1+ x86-64

python_ucto-0.6.7-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

python_ucto-0.6.7-cp39-cp39-macosx_11_0_arm64.whl (15.4 MB view details)

Uploaded CPython 3.9macOS 11.0+ ARM64

python_ucto-0.6.7-cp39-cp39-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.9macOS 10.9+ x86-64

python_ucto-0.6.7-cp38-cp38-musllinux_1_1_x86_64.whl (26.0 MB view details)

Uploaded CPython 3.8musllinux: musl 1.1+ x86-64

python_ucto-0.6.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.8manylinux: glibc 2.17+ x86-64

python_ucto-0.6.7-cp38-cp38-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.8macOS 10.9+ x86-64

python_ucto-0.6.7-cp37-cp37m-musllinux_1_1_x86_64.whl (26.0 MB view details)

Uploaded CPython 3.7mmusllinux: musl 1.1+ x86-64

python_ucto-0.6.7-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.7mmanylinux: glibc 2.17+ x86-64

python_ucto-0.6.7-cp37-cp37m-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.7mmacOS 10.9+ x86-64

python_ucto-0.6.7-cp36-cp36m-musllinux_1_1_x86_64.whl (26.0 MB view details)

Uploaded CPython 3.6mmusllinux: musl 1.1+ x86-64

python_ucto-0.6.7-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.3 MB view details)

Uploaded CPython 3.6mmanylinux: glibc 2.17+ x86-64

python_ucto-0.6.7-cp36-cp36m-macosx_10_9_x86_64.whl (16.1 MB view details)

Uploaded CPython 3.6mmacOS 10.9+ x86-64

File details

Details for the file python-ucto-0.6.7.tar.gz.

File metadata

  • Download URL: python-ucto-0.6.7.tar.gz
  • Upload date:
  • Size: 104.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.5

File hashes

Hashes for python-ucto-0.6.7.tar.gz
Algorithm Hash digest
SHA256 512fb192455b5ccec559159bc62358a68e9fc4d48ae644be9bc0d7b37378ee3a
MD5 c7aa0a2e0460f81ba52c2ffd846e322d
BLAKE2b-256 29aaf027fe0f069c51fd5701172a447943af76a4cfe39a7f9ac98814ad15024b

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp312-cp312-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 75c2814548684701346cdde7d442868553a68758e9cc94aa072fa6180a801593
MD5 97af25262514d0e8168877fe88cf4dcf
BLAKE2b-256 b4d987f0c20ee5c597bbabfe0ed987fb4100c3c8620071851103f832a0b33555

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 ec97216860d457fc7c5909c1ef775c8b4f2d54702041484efe6b4a1b796f0feb
MD5 1400db93fcc3d86d25685c2f7ae18536
BLAKE2b-256 82c78dbea1c60c64ab308345fc86b9797c87496d716cf64b8bc9cbf1bc2497f1

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp312-cp312-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp312-cp312-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 903991f4445204d81ab34478dc420ac8139bef0ea9d5a98cb7c4eae85502fa7b
MD5 b82d59a15a6b7a2df89b753028cead47
BLAKE2b-256 0a9b68cc6bd92421e7934ef8451a10616df3cd6595afbbf6e40f10165380aa86

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 4276dfa686afb5e13f9a60139a4827c1084da9c93320b4e1811f192ea0d1a7b5
MD5 70244c5cac7e96b9138beb09447c9d1e
BLAKE2b-256 6da5e52345bc28d317f1e07f760652a3440438b5c767807407392d2cc596e663

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e7aa7c96616a4943833c029b8dcfa7b84f5f49c4ab5c7378962ba6197e3d6eae
MD5 cf9b4fd9f8ae2636505882aa60278223
BLAKE2b-256 69ce830b8ac02580df22348eb6c3f0bf875da9511f6d8c1cc17b8dd3d18b2b32

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 eff7a87da114d242bc8133030793611e954cdea42070e04260026e0e36bedf23
MD5 5bfd42dc824b4a011f78ce48d69cd730
BLAKE2b-256 a27fbbed634964eb89ce4095ffb3042980b77b198c48d0ecbbf28872733ca4a9

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 6ff8b7b02d3062c7f445b4c5c19ab35254beca677f1edf38948a76710b35bbc6
MD5 5aa437b9a20dc099df7b514eaa733324
BLAKE2b-256 7187c953313daa1cef14f173c756b75c12574cb4ce8fae69c7cbd9034ec60a36

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 6ac68722012ec18bfb034bb87ce1e1f1fcfd437e749322a52805532045165641
MD5 b0685031f8c0c62aa3e5179d797dd5ae
BLAKE2b-256 f79396f180b26cda8efdb103f851149d8105807f5676cfd61288c969450156c1

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 dcaef3972756ea90cb146b608c3454e314c2c44d8123486fa085eba257315d3a
MD5 17d4402094c5ee6084a471926fa82c5f
BLAKE2b-256 40de6149dc61f6893fa8f3953419caa693f16339a93a9d9a640ecba637ff2808

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 3ff20f4b70a4028255d11992930616a64f14f2aa66c29c5634c90f0af4a68b13
MD5 0429d75200e02a18c029f81e659462f8
BLAKE2b-256 1d14ffc23f6e01ae807315e91b1175397aedc30b106c9527044fe54c30c881a7

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 f490a6c9affc8828cd5045d2c10596486ab4766dc8c01d36c5fb97d3cbcf8c08
MD5 aff3b426f91a50455328ba6a25db16a0
BLAKE2b-256 92ae44d06147e2316f86e75edf082c8fdafebfa421580deb1838df17785918cf

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 479a08b41d990549c454a7d60998ec40990e60f6330915bc132e48d1530eb736
MD5 4a4a0671aa9c0e53d4a8180577e2cc55
BLAKE2b-256 629053718878bfc2a716cc7af1d0170c85e9fba48bc9693c1bf7513ea9127852

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 3bcb4d50e7faa73fe22658426dab89c4e0ecab0d1c6e8c9aaf39c3b8ebe6ef75
MD5 7beaa8c204f255a508258d08ec555f88
BLAKE2b-256 6cb5e48f0c9def21d227823913fa7f4b057f6a23e3ad3107ef3bbe8521485103

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 286818db3d6d1db08c7f6415fd7b73a32b06a95b903165306bbfc2e2a65266c4
MD5 480373d5c90be7d741706df251666235
BLAKE2b-256 99923e836f3c1dd3af2caeb0d2522c06a7598ae3249d72e5befd4062b953c91d

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 cad15c50b4b89625a0e4adb7d9bf92dbaeb2d17b7ee6d865331980a245796837
MD5 f6948f08eaed4071d2484357ac880d6b
BLAKE2b-256 9bbb12fb90a36e0142ea297bb406368e189b929fe4934c528bced8aaa0cd6542

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 49b9bb8d7fb275dac9fc2c034bb4de4765e9a6119df2e6bf658a0d7463e42799
MD5 e3325d811cd4574be647adfa47cbafcf
BLAKE2b-256 aff9c7a62034d2347ce302f245f19f5b4b1bb4ccf021d066eb15cd02496c37f5

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 93201e1b5965195db5b2529068ad627effab0b84f7503e4cddfa36e0b3e8ca8a
MD5 28ccc523838b4dd57b3f1952a0ab787b
BLAKE2b-256 4cd56626559887d3037a2ddf29d75fba25ed94a8446b48b3a3dea8027a114ad9

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 730d44e0fe8e882bc3307d99950472632c70ba422e2c66f694134c93d5148f86
MD5 0cd03e8e6499243be3fdb61945891e64
BLAKE2b-256 6a04544dca2b2f7e57bd1a1283c0fee7c75c210795e3413574abd6609d657497

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 d9edb110df936b261f6fd8f53b634739bf5ff96531baedec92e9c96ee96467a2
MD5 9270bf9d1165b7b4befff14b31bf803e
BLAKE2b-256 8f1f8a2a84555d86ec9469b92386544b4297d19e175c4d58d49fd7204555d0af

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1995ce9bd3eac14ea9287fb6011e6f2bfb5976a31760bd417c6ab3710185a916
MD5 4f7056a244403c8fe686c43b161698d6
BLAKE2b-256 da8ca0cf748429ea2d4d566f85e79bc6cc91436a7df38f455b070df83dbca4af

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 47e22ab48ff83b8d78f63a93dc23ead1467b038b7d890347075a5ff7fc8cc469
MD5 9f88545d19eb96e749d1e6025c1c16ab
BLAKE2b-256 c8c6dfa2ea4ed40bb5da531a271a27b76907c2fdabb7498e8bb4493f9f62f95c

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp36-cp36m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp36-cp36m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 8befcc84c6f6bce9f181cac7f5aa8c3d19a94cdf37164f1848a9172f2192b136
MD5 060ee0a2653c200bf5f6778e6cc2609b
BLAKE2b-256 08c2eef16414931bd25de5f9ebd2175cc93de48faa31e5f4f48af0247820f327

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 10df0c8e8fce35744c45257d3dc4094550cecb683f5f7ceeb26592e61d51c896
MD5 710486d9c298283d867d3bd592b6730f
BLAKE2b-256 e3db0b8127a0ed6513bd215936b30775874739c453a96f2ec308e08b023f32cb

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.7-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.7-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 232eb038a403bc91a912c24419929196b3011a9a810dee5f0f7be964de98021a
MD5 f34c0334d8425cbe58bafa709e88a7e4
BLAKE2b-256 771a202cc7833d70533cee8f46ae183b3a496057bb0696f8906460de258de1ed

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page