Byte-Pair Encoding tokenizer implemented in Rust
Project description
The author of this package has not provided a project description
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
smoltok-0.1.0.tar.gz
(208.0 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file smoltok-0.1.0.tar.gz.
File metadata
- Download URL: smoltok-0.1.0.tar.gz
- Upload date:
- Size: 208.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: maturin/1.10.1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9f9d8ca65d530609d7b0e6b76c19c4a64bc2f75997fab4973938a71d447ddc19
|
|
| MD5 |
838a30287c37ae4939cea39504409768
|
|
| BLAKE2b-256 |
a0003b9ef8e43d520c27c3537978a2a3a1824845acf57b529034a08fd1e762d6
|
File details
Details for the file smoltok-0.1.0-cp38-abi3-macosx_11_0_arm64.whl.
File metadata
- Download URL: smoltok-0.1.0-cp38-abi3-macosx_11_0_arm64.whl
- Upload date:
- Size: 970.1 kB
- Tags: CPython 3.8+, macOS 11.0+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via: maturin/1.10.1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
5a3266a438776022b90aa8f6aa0230593c4d9c38663e6de35c52333a77d0a08c
|
|
| MD5 |
3dee37f8b34385823382555459013bb7
|
|
| BLAKE2b-256 |
7ccd0dc2b51a7007b6a32fa4604ff9447561c544d1209ede6190db8286a8bd38
|
