Tokenizing and processing text inputs with transformer models
Project description
A package that provides functionalities for tokenizing and processing text inputs using transformer models and other NLP tools.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tokenize_text-0.2.32.tar.gz
(60.4 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file tokenize_text-0.2.32.tar.gz.
File metadata
- Download URL: tokenize_text-0.2.32.tar.gz
- Upload date:
- Size: 60.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
82f9e433ad76246cbcb915d9eb33c32ab201b8fe3f616b857a1aea5e71a31604
|
|
| MD5 |
fce8b788882829dc2b8978ff1aa7c6e1
|
|
| BLAKE2b-256 |
e5fbd5027fe22c54580709eb25f3b80df1978a14477b90b5f5c5b4595c841dba
|
File details
Details for the file tokenize_text-0.2.32-py3-none-any.whl.
File metadata
- Download URL: tokenize_text-0.2.32-py3-none-any.whl
- Upload date:
- Size: 59.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a17a8e3a50d82830053026215c054ce1a1706796ebfb91a8d88644300ede0d9f
|
|
| MD5 |
008c8101944f1d8ab6baf2440ca3a180
|
|
| BLAKE2b-256 |
80d93adae72518280a3420ac191af5aa04d118c0f0cd3faf66237cc479eba8f3
|
