Skip to main content

Add-ons to the huggingface `datasets`

Project description

datasets2 provides add-ons to the huggingface datasets.

Example usage:

# datasets is just the huggingface datasets
# load_dataset and save_to_disk adds parquet support to datasets.load_dataset and datasets.Dataset.save_to_disk
from datasets2 import datasets, load_dataset, save_to_disk

my_dataset = prepare_my_huggingface_dataset()
output_dir = "my_dataset_dir"
save_to_disk(my_dataset, output_dir, parquet=True)

load_dataset(output_dir)  # automatically infers if the dataset uses parquet format.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datasets2-0.1.1.tar.gz (4.8 kB view details)

Uploaded Source

Built Distribution

datasets2-0.1.1-py3-none-any.whl (5.7 kB view details)

Uploaded Python 3

File details

Details for the file datasets2-0.1.1.tar.gz.

File metadata

  • Download URL: datasets2-0.1.1.tar.gz
  • Upload date:
  • Size: 4.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.11.3

File hashes

Hashes for datasets2-0.1.1.tar.gz
Algorithm Hash digest
SHA256 f4f6956537826214e98bcbad0f570c4b8a0dbca1445e3be522b6a06eeb89c07f
MD5 b2eee32f6e2c262a17daba1aee9bd3de
BLAKE2b-256 e1382afbb66a72c7ca53d2adc09503ea20a9f74c7f009ff9130313151dfd831d

See more details on using hashes here.

File details

Details for the file datasets2-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: datasets2-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 5.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.11.3

File hashes

Hashes for datasets2-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 772d00d7799fe2142d34ad067d34fa57f5599dbcada1fa2a1d660ad614088345
MD5 e4c6198b80eef8ecb1dc02fc1f0f9d54
BLAKE2b-256 5bff9b4dc54201a9e99784039826809f9d05dbd3c261d051f2c0dfb6a00184f3

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page