Skip to main content

A simple but handy Python library to save [PyTorch](https://pytorch.org) datasets in (SafeTensors)[https://github.com/huggingface/safetensors] format.

Project description

safetensors-dataset

safetensors-dataset is a very simple and tiny package adding support to efficiently load and store Pytorch datasets in the safetensors file format:

import torch
from safetensors_dataset import SafetensorsDataset

dataset = SafetensorsDataset.from_dict({
    "x": torch.randn(8, 12, 3),
    "y": torch.randint(12, (8,))
})
dataset.save_to_file("test.safetensors")

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

safetensors_dataset-0.0.1.tar.gz (8.2 kB view details)

Uploaded Source

Built Distribution

safetensors_dataset-0.0.1-py3-none-any.whl (8.7 kB view details)

Uploaded Python 3

File details

Details for the file safetensors_dataset-0.0.1.tar.gz.

File metadata

  • Download URL: safetensors_dataset-0.0.1.tar.gz
  • Upload date:
  • Size: 8.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.0 CPython/3.10.13

File hashes

Hashes for safetensors_dataset-0.0.1.tar.gz
Algorithm Hash digest
SHA256 ce3bdf176df208622321e9cbf0fedec0c77b3c1a011e09cdfef9f050cbc75297
MD5 886aa7897c85ce70a9204519cfb87e4b
BLAKE2b-256 e9cde18c2065188f2e4f8cda4e4b8a93197bdb7bf791f9b9c0a3e727f35e9696

See more details on using hashes here.

File details

Details for the file safetensors_dataset-0.0.1-py3-none-any.whl.

File metadata

File hashes

Hashes for safetensors_dataset-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 18e74d5c4c2bb62a45154f9beb6b1779fa179dbc24edb21d3a935f4b9f7365ae
MD5 e18f9ec3ecde1ac9eec3af7d2fa6ae66
BLAKE2b-256 24223fc0a5be1c9dd2229b689068f9068c8f3edf1bd24a8490a622456c32fb4c

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page