This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is a regular-expression based, extensible, and advanced tokeniser written in C++ (https://languagemachines.github.io/ucto).

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for proycon from gravatar.com proycon

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: GNU General Public License v3 (GPLv3) (GPLv3)
  • Author: Maarten van Gompel
  • Tags tokenizer , tokenization , tokeniser , tokenisation , nlp , computational_linguistics , ucto
Classifiers
Report project as malware

Project description

The author of this package has not provided a project description

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for proycon from gravatar.com proycon

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: GNU General Public License v3 (GPLv3) (GPLv3)
  • Author: Maarten van Gompel
  • Tags tokenizer , tokenization , tokeniser , tokenisation , nlp , computational_linguistics , ucto
Classifiers

Release history Release notifications | RSS feed

0.6.10

0.6.9

0.6.8

0.6.7

0.6.6

0.6.5

0.6.4

0.6.3

This version

0.6.2

0.6.1

0.6.0

0.5.3

0.5.2

0.5.1

0.5.0

0.4.7

0.4.5

0.4.4

0.4.3

0.4.2

0.3.0

0.2.4

0.2.2

0.2.1

0.2

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

python-ucto-0.6.2.tar.gz (79.0 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

python_ucto-0.6.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.8 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

python_ucto-0.6.2-cp311-cp311-macosx_11_0_arm64.whl (51.7 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

python_ucto-0.6.2-cp311-cp311-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.11macOS 10.9+ x86-64

python_ucto-0.6.2-cp310-cp310-manylinux_2_28_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

python_ucto-0.6.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.8 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

python_ucto-0.6.2-cp310-cp310-macosx_11_0_arm64.whl (52.3 kB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

python_ucto-0.6.2-cp310-cp310-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.10macOS 10.9+ x86-64

python_ucto-0.6.2-cp39-cp39-manylinux_2_28_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.28+ x86-64

python_ucto-0.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.8 MB view details)

Uploaded CPython 3.9manylinux: glibc 2.17+ x86-64

python_ucto-0.6.2-cp39-cp39-macosx_11_0_arm64.whl (52.8 kB view details)

Uploaded CPython 3.9macOS 11.0+ ARM64

python_ucto-0.6.2-cp39-cp39-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.9macOS 10.9+ x86-64

python_ucto-0.6.2-cp38-cp38-manylinux_2_28_x86_64.whl (25.9 MB view details)

Uploaded CPython 3.8manylinux: glibc 2.28+ x86-64

python_ucto-0.6.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.8 MB view details)

Uploaded CPython 3.8manylinux: glibc 2.17+ x86-64

python_ucto-0.6.2-cp38-cp38-macosx_11_0_arm64.whl (52.0 kB view details)

Uploaded CPython 3.8macOS 11.0+ ARM64

python_ucto-0.6.2-cp38-cp38-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.8macOS 10.9+ x86-64

python_ucto-0.6.2-cp37-cp37m-manylinux_2_28_x86_64.whl (25.8 MB view details)

Uploaded CPython 3.7mmanylinux: glibc 2.28+ x86-64

python_ucto-0.6.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.8 MB view details)

Uploaded CPython 3.7mmanylinux: glibc 2.17+ x86-64

python_ucto-0.6.2-cp37-cp37m-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.7mmacOS 10.9+ x86-64

python_ucto-0.6.2-cp36-cp36m-manylinux_2_28_x86_64.whl (25.8 MB view details)

Uploaded CPython 3.6mmanylinux: glibc 2.28+ x86-64

python_ucto-0.6.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.8 MB view details)

Uploaded CPython 3.6mmanylinux: glibc 2.17+ x86-64

python_ucto-0.6.2-cp36-cp36m-macosx_10_9_x86_64.whl (15.6 MB view details)

Uploaded CPython 3.6mmacOS 10.9+ x86-64

File details

Details for the file python-ucto-0.6.2.tar.gz.

File metadata

  • Download URL: python-ucto-0.6.2.tar.gz
  • Upload date:
  • Size: 79.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.9

File hashes

Hashes for python-ucto-0.6.2.tar.gz
Algorithm Hash digest
SHA256 dbd0dc129c7d745f05913c83368a55ea018956ebcf4ded27c027bc9b6ae12b5f
MD5 6e3a73f5e2671cb38e5e4d03156cb020
BLAKE2b-256 db3768ff14365586dd229acc5366297da2cf4613ada8afa0f8ae9f9e3152356f

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 255a3a39c4aa645880815d0df2572d4b5f661ee3bc9add32cb0c11a77a308484
MD5 bbbdf05b18a322aa0dce1560095fa848
BLAKE2b-256 585f94ee0e27ef2bc8b3587d1f1b51f29be1203a8a5e1a320b34593ae397943b

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f9b24cd4a78ae948c6736931a7166c14641cef2a16e763c7401c6a04dfd5ddc6
MD5 4995ae3dc56b807d3bc45fa407cbe6fe
BLAKE2b-256 8b3798b195283d8357383e57ac36d593f0a667ea5cb3946d92f7e603feac8b90

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 a5fc01229ddd54148a15e56c33aea8404d25b37bd3ac3c503b5bd73b69afa2a9
MD5 691d359e725f1a15e75fd142dbf85434
BLAKE2b-256 cdda88359f5b1eb939a75411cba1f2340c4798bef98c6575bca6727b6dbe6002

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ba7653bbd0a34a0ed6c2a3da3f0e70816f06128d7143a8f2c7e2428578acabc3
MD5 1f49bfbc4ca4dcc9a8b3b68cc1ff05be
BLAKE2b-256 782445cd382d13bfa95ee72150a29aefd7778c5c4d94d82afc712002ad2c07c2

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 bbe16211624a6b06bbff747c57790e8c9c40a7aeb4c5f2c57132793e283fa4b7
MD5 76f554e03843c54f0689d5499882f1e1
BLAKE2b-256 8a78f217c5e4c860dee9d83686ea159641cfd667bc1288afb295cacc271a05fb

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 711fef4c8c216ebcbfdaf46be4decd2c68e474469e45b1298a400721e5ede301
MD5 8bf81483d6ba686ef6efdd7f5401860a
BLAKE2b-256 3ecd3dcdbdb8ad1fe7ba3a2064649ec0cbbb5a9244a7be3f7ea990829ca45208

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 f717936e13d1fae9bd5491bcd3c73d8beee3cb200b73df700f36f0f26b0ccae2
MD5 4688b1d3a790c1d9bf1096c73ae1cd7f
BLAKE2b-256 5347a61000476830f3c3770861daab63bad9cdd1f4253c6bdf88294771b37194

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp39-cp39-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp39-cp39-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 e9f83491c757d4bee0ec978c2355753fd8465a48bc7cd231d6d8b4bb9673ee68
MD5 49a5ebd8dc9507494c7e4216b1d2f363
BLAKE2b-256 af7af8029e0cbc84177cc5c189bc68f5fc59a00e020103d6fcf6e63d9e1c25b1

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b5c16d1068c9c419082b57e3a95086dcb427d1f97de689f75691378bc7ba9597
MD5 df293a853d2e7f0c63d84527672d0e24
BLAKE2b-256 2804e7f4148b8781493520571dd15ce7fe21565362d3c0488aefbe1d5d4ed4f4

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4af5a29efa8dbc97841d1192ac0a702864ddb056b57959fe358c18c526bde4db
MD5 d152bfd6fe50ec7c29fd4ab33b460e6d
BLAKE2b-256 e1922e42f190d9c3e30c476ce4a78931c3c01753f0a92cbe179f68765343decf

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 0746cb14c20c430b2b2b806cde467edaf29b7b7406845b345d1765e11511e7eb
MD5 7a987fc4213141e383a1810be451423c
BLAKE2b-256 c5101376d1db9e53ba14ffe67b2eac2f6efd2babf6286a586e8eef5d14e061d8

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp38-cp38-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp38-cp38-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 eefaf5cd6d4066b3a593b5534deff3f25fe0bf544916d3eb61c6678cea7c0fd7
MD5 eba79b413bc1819a087a25ba6b33593c
BLAKE2b-256 8c97d4df46ad386e25e0808117bf0b4aac6ee007c3ef24eaa756b20a1ca1ee93

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b95a12bf9ded5a145ab1d54d0d777c3392f868f6e0db4c5057506249eae44d8e
MD5 aba81b09bb76cb09d8ee147ef4abe45d
BLAKE2b-256 0f74edcb002bb735184e5441a6faa58490c60b9f0b374afd0984c1fb93e8be51

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 79894bdf139c3e72d2c0015f0a521adca0b823856eca3300b08f6c79e78b2d5b
MD5 7a23c23dcc1b49a03da34dfca4d665be
BLAKE2b-256 603226b1119231f278e2c1eba6d58180aa6ed315608db06cde06ea747fa608d0

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 82a08343470ff7332fcdaedec3a1b7740601563a23f283d98e7ec183a70e57f3
MD5 69cb82778f403caf27e562ce3f8fc854
BLAKE2b-256 cbbb53fb48e4e2301c36098a5818940da2f905081f93839bfac284151754b05b

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp37-cp37m-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp37-cp37m-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 dffb7793936d1fa4cfba727c0db165ce1babeb50baac6f2e88225ec4b8d85174
MD5 8f9c9c0bc88dfa0e551290b831911e75
BLAKE2b-256 8bdd60851b5c80dc18805f2426457dbd5ac0a624a79272212436d3676d3c8b8d

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 4888a14d74fa4c51e5340c644622315609a56017d8dce90d4e130458ec0a3b6a
MD5 b83df5fd11fac5e713a3d58a32c7a090
BLAKE2b-256 3623e8c980ff086caf20e8a2d56477997091d85e57df2fedc20c3d15b0fd0935

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 e67ef5ac3430a95834b32131caccf75a5ef7180aba78eb5b86e6ab98a7e02bbc
MD5 91abd533bc57b8bf7c5d434960faf1e8
BLAKE2b-256 38575ae160ace7af46980c4be0a8c7aff8fc51a22a6bf8ef643154c2531677bf

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp36-cp36m-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp36-cp36m-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ef148b00d6dc4e0436114828e405933574c610067ab29c839ea2f2e53c540119
MD5 d3cc8384d50d053338486da0f3e6988d
BLAKE2b-256 3d06e7c53376dc864ead8a62261d3a96c381f17e16598584d48f01caf346a3aa

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1aa994070063bf05173aa3ac48a797da030542e1bd93ca03ecd1ed3f84d6360c
MD5 f6d9c0cebe7c1d4f715335319658f03c
BLAKE2b-256 a92ce76711683a3c0ac86b1506972c824672370f9f9535404ac340982e149145

See more details on using hashes here.

File details

Details for the file python_ucto-0.6.2-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for python_ucto-0.6.2-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 67f88476226d1f906b319c78588c470f72c8d199d1924e21ede985e71c8c0ee8
MD5 624f653741265be5612e546cec6fca72
BLAKE2b-256 cc9256bbdfddc3b59e96f9ae70abf5c4838db3268151a2ebce206220581d9f99

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page