A streaming tokenizer.
Project description
pyTokenizer
A streaming tokenizer.
Contributors
- Patrick Lehmann (Maintainer)
License
This Python package (source code) is licensed under Apache License 2.0.
SPDX-License-Identifier: Apache-2.0
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pyTokenizer-1.1.4.tar.gz
(5.2 kB
view hashes)
Built Distribution
Close
Hashes for pyTokenizer-1.1.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 117de4342add39128cd38202f4af098ef6ee1938e6a7fd749f27093e6d23daca |
|
MD5 | 2bd4c84e30a98597b5d87ead2128075b |
|
BLAKE2b-256 | 97588b6d3189bd4c10d1092763f53ed42bdba1bfea6374f5e66dd376d21cf7ed |