Tokenizer for semiformal unicode text using TR-29 segmentation

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for albarrentine from gravatar.com albarrentine

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT License (MIT)
  • Author: Al Barrentine
  • Tags word, tokenizer, lexer, re2c, fsa
  • Requires: Python >=3.7
Classifiers

Project description

semiformal

Tokenizer for semiformal unicode text using TR-29 segmentation

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for albarrentine from gravatar.com albarrentine

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT License (MIT)
  • Author: Al Barrentine
  • Tags word, tokenizer, lexer, re2c, fsa
  • Requires: Python >=3.7
Classifiers

Release history Release notifications | RSS feed

This version

0.7.0

0.4.9

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

semiformal-0.7.0.tar.gz (593.5 kB view details)

Uploaded Source

Built Distributions

semiformal-0.7.0-cp311-cp311-musllinux_1_1_x86_64.whl (3.4 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

semiformal-0.7.0-cp311-cp311-musllinux_1_1_i686.whl (2.5 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ i686

semiformal-0.7.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

semiformal-0.7.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl (2.5 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ i686 manylinux: glibc 2.5+ i686

semiformal-0.7.0-cp311-cp311-macosx_11_0_arm64.whl (598.3 kB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

semiformal-0.7.0-cp311-cp311-macosx_10_9_x86_64.whl (522.5 kB view details)

Uploaded CPython 3.11 macOS 10.9+ x86-64

semiformal-0.7.0-cp310-cp310-musllinux_1_1_x86_64.whl (3.4 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

semiformal-0.7.0-cp310-cp310-musllinux_1_1_i686.whl (2.5 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ i686

semiformal-0.7.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

semiformal-0.7.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl (2.5 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ i686 manylinux: glibc 2.5+ i686

semiformal-0.7.0-cp310-cp310-macosx_11_0_arm64.whl (598.3 kB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

semiformal-0.7.0-cp310-cp310-macosx_10_9_x86_64.whl (522.5 kB view details)

Uploaded CPython 3.10 macOS 10.9+ x86-64

semiformal-0.7.0-cp39-cp39-musllinux_1_1_x86_64.whl (3.4 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

semiformal-0.7.0-cp39-cp39-musllinux_1_1_i686.whl (2.5 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ i686

semiformal-0.7.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

semiformal-0.7.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl (2.5 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ i686 manylinux: glibc 2.5+ i686

semiformal-0.7.0-cp39-cp39-macosx_11_0_arm64.whl (597.8 kB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

semiformal-0.7.0-cp39-cp39-macosx_10_9_x86_64.whl (522.2 kB view details)

Uploaded CPython 3.9 macOS 10.9+ x86-64

semiformal-0.7.0-cp38-cp38-musllinux_1_1_x86_64.whl (3.4 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

semiformal-0.7.0-cp38-cp38-musllinux_1_1_i686.whl (2.5 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ i686

semiformal-0.7.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64 manylinux: glibc 2.5+ x86-64

semiformal-0.7.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl (2.5 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ i686 manylinux: glibc 2.5+ i686

semiformal-0.7.0-cp38-cp38-macosx_11_0_arm64.whl (597.8 kB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

semiformal-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl (522.2 kB view details)

Uploaded CPython 3.8 macOS 10.9+ x86-64

File details

Details for the file semiformal-0.7.0.tar.gz.

File metadata

  • Download URL: semiformal-0.7.0.tar.gz
  • Upload date:
  • Size: 593.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.4

File hashes

Hashes for semiformal-0.7.0.tar.gz
Algorithm Hash digest
SHA256 97d7005e099d4d2d8c9f7fd609aeed2238d528ecf4545563ee9e4bb507593a44
MD5 8cd07dc4ee0ba63c73ff8439260d99ae
BLAKE2b-256 a7c7e2904b25f25c1ea5cfffa3f4385a131a956b22c3831015351f682e196a13

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 1b0aff4055baebdf0f0c9f249915f88440121956bb91caa132217140e93c360b
MD5 e280494b52f0d6dc707935146abfd5c2
BLAKE2b-256 08bead2aeb674cef3f49a7a9a54d2cef913253faa026c9512fd124974d4a3447

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp311-cp311-musllinux_1_1_i686.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp311-cp311-musllinux_1_1_i686.whl
Algorithm Hash digest
SHA256 291a6cb5baa8807ae9b4f8308ec2ddb6a26a1c852d42241346cf4c6bfa67b8c4
MD5 20c4557f6d22521ca7491256be9710a7
BLAKE2b-256 b84eb350b2aff85466e16e2c9e7458e27bca98773d34e9147f6222ce41723d19

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 76e70321903ca45f35ea2b30b53e784779d85e5ea4110e30dc08180a0978a722
MD5 0303e302052d43e7d26f0c8c5824c806
BLAKE2b-256 1d42040cce79bb099f6ba7e122875cbca0fe523f3cc3b7b470765ad4c979d6a9

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 5c39d9fb8ca07eb8f180b5ced48f149f2550b15f347669e1b8108188e126d347
MD5 db9a0d1b699c4aae0d7d483c3f6b0fe9
BLAKE2b-256 873a78e2171c17f0afa9758e7502dcadd99cec448db9a402d97727fc237d4998

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 fe87f8b43e73ce1fac34059aebaabdc1d01380f390d49bec05755770701de243
MD5 4162e598119353598ef00a8d4790569d
BLAKE2b-256 2d82a2c5a34daffffad0f7d7d1a1e0756eddfe0df4e6448b6ace102b0d4ce5d1

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp311-cp311-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 36f151fba2a835f0e0d95623bf9468c9b28df19578f976c361bb263edcb941de
MD5 446f22fab65d2bbe2c7401a1a5fe9c74
BLAKE2b-256 58d82c571347c10cf1ac825fc01d263287cb0b6b5432adafaf81defa6220d441

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 bc69c5a4c8bc189c257838a221f1971acb76690fb24c2e7e0dc4447bd51456ac
MD5 148af4e60b7c6e0082fa8ade176688a4
BLAKE2b-256 4504f3a69784721d68ea9875c35edf9f91b46ada1d53f441629d1c28b3b3dd85

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp310-cp310-musllinux_1_1_i686.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp310-cp310-musllinux_1_1_i686.whl
Algorithm Hash digest
SHA256 119dc48ec732593379d54a77859cffa98d877d138f472e60905717b58e379fde
MD5 d1bb8752cfc8444ca59a13104768ad98
BLAKE2b-256 70b0db735137d665b591db029112cf200273bea1159e82998c28a7160363d73b

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8e0f5de066b230b1f2d8a4157e787ac81d1b5d9675394b0b25936c878398428b
MD5 b51599bd994da43fca1dadf584433105
BLAKE2b-256 6601284782a2c42a3ffc38c6da4aa952f34f38863c1bd52d4bde4a72d5ce1e1c

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 e20667ad055d256de670c7fededfa2f08faee81eb858955954dc146648e4008f
MD5 135a345fc0b0126236ffbc58412ec762
BLAKE2b-256 1d6878621d5ad12119194eb689c578ba7871fe250341d5a34f0103a36c09cbb9

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8c0c432d04ecce24c6a5fa6bad688b3868c73a8f344f9961ed575ed00d9cee4f
MD5 f61c2a9ab827a187269ff5f565fdb50b
BLAKE2b-256 6fcb5abd26780cdc3dee021e2077e20e6d7190ffbc3d9091141f9b2b848c6192

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 e770328bc391acce101c01e1f26f16ecdb6c8e853de1fd2873cf63ad49cb97eb
MD5 192c303c4f3017fc2aa198304bc8b701
BLAKE2b-256 51c44e2c6dc299f9d608589e97bf28295f46411a017f3360cca0faa537657203

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 080ab989df7510f13e13f544c7474a188678d0f49944c57cd4c8e3cfb8de624b
MD5 1554c37970051205c145c268c7afe8a5
BLAKE2b-256 17cdaf0c8e531d817167e39ebddae70367cab66af03df5bfc260488dac41d4ed

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp39-cp39-musllinux_1_1_i686.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp39-cp39-musllinux_1_1_i686.whl
Algorithm Hash digest
SHA256 3fb8ba0f6da7c7e68d91d232c189262d5484a187d4e3953e6f6969759d61524b
MD5 9261a7ad104fcaf7111c8be4e332d0f0
BLAKE2b-256 152bb8292d8e903890bc76e0317afbf870c8e555991667e0778d341ef08e3392

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f969eee05bee988c2edf16e791fa9aa6db61e729d6ba4f14833bfa1b109f941d
MD5 6e060e6f355ced05c36fec0b08dd242b
BLAKE2b-256 b48e3d016dcbdaee683fabccf040e18747b16f55229dcd2d2aa6b33632b9c663

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 d0db49cb254ebd62107b55e02dbe0db59c6911e55bd96e9f30a244bd71144897
MD5 c77e09fefb9c6ea86070abd9cf09c19b
BLAKE2b-256 3c277bd3b156e7e0ad172fc7a22dbd09a8cbe5679ce08a6b962a6efe0594c2bb

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 015991276583c3cbb6a8a67feefb56c6e0047289f2c76acca72f7bbd82730d6f
MD5 7c65342fb19b6d3a70ff99ab2a37d178
BLAKE2b-256 f908f6b3e0f38cf0fc02e0512f576fa5ae2a388cddb8f15c9f42a32a04983fd6

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 87b28bfd58e5e7d8101b3a42396b871543e848d1433b0bb40b35878c2698e04c
MD5 0e97602f6029460e40ecc9fc964155fe
BLAKE2b-256 5d632950ef3a72499f6e474350e89f95918a3941b2ea727cadd874da3877ac9c

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 b80df6e4e1e8afd570b16506faa78d6203358cc83c0d55397bdd60770a833e0a
MD5 5d6e0b79ea8e3d3ed7a4a5acf4b586c2
BLAKE2b-256 add45f7828d170484ef3c48f60a72fd51b36ae40912611c2253dd1cbafd0d309

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp38-cp38-musllinux_1_1_i686.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp38-cp38-musllinux_1_1_i686.whl
Algorithm Hash digest
SHA256 e4cb8b75750a355874e6eebd1f047a46af15da0eae85db5f884af72ab1f47988
MD5 f09f9fc2d0ef3c0a64160c9e53b1a9ca
BLAKE2b-256 9b15e41a6ee0fa18569c6df543f2ec117a8b8c890fcc81cbdee30a2291cd0f17

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 4c61727169598f99c6b7af3dcce2088ba138599da4a5ed7c644ad44431478144
MD5 69cf8d83a1a8f23fba34ae7eeb3e65c8
BLAKE2b-256 efc7d1de18a6faef9bfbbf572898b4e7e0f7739f14268c7df5793710d055711f

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 165b4c3277ab7276e254c5bd1377baea6c47abc845c5f0e0e2e38821935902bc
MD5 a0a506e7b5304d41ab915962b2f777b1
BLAKE2b-256 5324d1b7c583350a5d978d9034c79119f5bf710e9e3139448c0d0c7fb1c4dfa4

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 dbdbb6edc37a54f2852f92cb81ef36aa3c1cfb0da5ab6ae22dcd3f94b05493e9
MD5 02ac2eceed2ea0638367e481107a54b9
BLAKE2b-256 a9571dc836900037373b70f2eb5cf67babd287828d427d81f5f001b369f735b8

See more details on using hashes here.

File details

Details for the file semiformal-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

File hashes

Hashes for semiformal-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 74685a4ea0999fe08021e72f86ed35ccfba5ad88a7b19f44e9d3e49895f81071
MD5 8683a0c7ae902bf0c6c50648ea1e1280
BLAKE2b-256 2660f9366cf5e576a00119da84726ab4dcf77e70839958b8cd7e11e3ea448491

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page