A simple but handy Python library to save [PyTorch](https://pytorch.org) datasets in (SafeTensors)[https://github.com/huggingface/safetensors] format.
Project description
safetensors-dataset
safetensors-dataset
is a very simple and tiny package adding support to efficiently load and store Pytorch datasets in the safetensors file format:
import torch
from safetensors_dataset import SafetensorsDataset
dataset = SafetensorsDataset.from_dict({
"x": torch.randn(8, 12, 3),
"y": torch.randint(12, (8,))
})
dataset.save_to_file("test.safetensors")
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for safetensors_dataset-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | ce3bdf176df208622321e9cbf0fedec0c77b3c1a011e09cdfef9f050cbc75297 |
|
MD5 | 886aa7897c85ce70a9204519cfb87e4b |
|
BLAKE2b-256 | e9cde18c2065188f2e4f8cda4e4b8a93197bdb7bf791f9b9c0a3e727f35e9696 |
Close
Hashes for safetensors_dataset-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 18e74d5c4c2bb62a45154f9beb6b1779fa179dbc24edb21d3a935f4b9f7365ae |
|
MD5 | e18f9ec3ecde1ac9eec3af7d2fa6ae66 |
|
BLAKE2b-256 | 24223fc0a5be1c9dd2229b689068f9068c8f3edf1bd24a8490a622456c32fb4c |