Tokenizing and processing text inputs with transformer models
Project description
A package that provides functionalities for tokenizing and processing text inputs using transformer models and other NLP tools.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tokenize_text-0.2.32.tar.gz
(60.4 kB
view details)
Built Distribution
File details
Details for the file tokenize_text-0.2.32.tar.gz.
File metadata
- Download URL: tokenize_text-0.2.32.tar.gz
- Upload date:
- Size: 60.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 82f9e433ad76246cbcb915d9eb33c32ab201b8fe3f616b857a1aea5e71a31604 |
|
| MD5 | fce8b788882829dc2b8978ff1aa7c6e1 |
|
| BLAKE2b-256 | e5fbd5027fe22c54580709eb25f3b80df1978a14477b90b5f5c5b4595c841dba |
File details
Details for the file tokenize_text-0.2.32-py3-none-any.whl.
File metadata
- Download URL: tokenize_text-0.2.32-py3-none-any.whl
- Upload date:
- Size: 59.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | a17a8e3a50d82830053026215c054ce1a1706796ebfb91a8d88644300ede0d9f |
|
| MD5 | 008c8101944f1d8ab6baf2440ca3a180 |
|
| BLAKE2b-256 | 80d93adae72518280a3420ac191af5aa04d118c0f0cd3faf66237cc479eba8f3 |
