Add-ons to the huggingface `datasets`
Project description
datasets2
provides add-ons to the huggingface datasets
.
Example usage:
# datasets is just the huggingface datasets
# load_dataset and save_to_disk adds parquet support to datasets.load_dataset and datasets.Dataset.save_to_disk
from datasets2 import datasets, load_dataset, save_to_disk
my_dataset = prepare_my_huggingface_dataset()
output_dir = "my_dataset_dir"
save_to_disk(my_dataset, output_dir, parquet=True)
load_dataset(output_dir) # automatically infers if the dataset uses parquet format.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
datasets2-0.1.1.tar.gz
(4.8 kB
view details)
Built Distribution
File details
Details for the file datasets2-0.1.1.tar.gz
.
File metadata
- Download URL: datasets2-0.1.1.tar.gz
- Upload date:
- Size: 4.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.11.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f4f6956537826214e98bcbad0f570c4b8a0dbca1445e3be522b6a06eeb89c07f |
|
MD5 | b2eee32f6e2c262a17daba1aee9bd3de |
|
BLAKE2b-256 | e1382afbb66a72c7ca53d2adc09503ea20a9f74c7f009ff9130313151dfd831d |
File details
Details for the file datasets2-0.1.1-py3-none-any.whl
.
File metadata
- Download URL: datasets2-0.1.1-py3-none-any.whl
- Upload date:
- Size: 5.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.11.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 772d00d7799fe2142d34ad067d34fa57f5599dbcada1fa2a1d660ad614088345 |
|
MD5 | e4c6198b80eef8ecb1dc02fc1f0f9d54 |
|
BLAKE2b-256 | 5bff9b4dc54201a9e99784039826809f9d05dbd3c261d051f2c0dfb6a00184f3 |