A simple tokenizer library
Project description
ReadME
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pre_tokenizer-0.1.1.tar.gz
(3.1 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file pre_tokenizer-0.1.1.tar.gz.
File metadata
- Download URL: pre_tokenizer-0.1.1.tar.gz
- Upload date:
- Size: 3.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.10.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
42e062296c2a0e89f6ff05dd90255c7814365e43b32b9d98899e0898356394b8
|
|
| MD5 |
f71e06bf71e77ed1e691890bae8652f1
|
|
| BLAKE2b-256 |
48b6f890cc84b7c5d2b0270af5ada69818c8ab8d6ff1008529045fea84c4b8a0
|
File details
Details for the file pre_tokenizer-0.1.1-py3-none-any.whl.
File metadata
- Download URL: pre_tokenizer-0.1.1-py3-none-any.whl
- Upload date:
- Size: 4.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.10.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
0cada1dc735cea123569bee08b0ade02b7448e1a2dbdfa6d77a4b4ed56c7e2c3
|
|
| MD5 |
99adf57adf1b5ce9abb5d6b193076ae7
|
|
| BLAKE2b-256 |
e55bf6f63ef3b594d89d8306696facab735de7d7bc56af0f43eaecd92b012338
|
