A suite of custom PyTorch dataset loaders for vision tasks like classification and segmentation.
Project description
torch-datasets
Note: This is a placeholder release to reserve the package name. A proper release will follow soon.
One toolkit for all your PyTorch data loading needs.
Still cooking — stay tuned!
🚀 Overview
torch-datasets is a unified, flexible, and extensible toolkit for handling dataset loading in PyTorch. Whether you're working with images, text, audio, tabular data, or custom formats, torch-datasets aims to make your data loading pipeline simple, efficient, and scalable.
🔧 This project is currently under active development — stay tuned for updates and releases!
✨ Features (Planned)
- Built-in support for popular dataset formats (images, text, tabular, audio)
- Easy-to-use wrappers for custom datasets
- Fast data loading using multiprocessing and caching
- Simple APIs for train/val/test splitting
- Seamless integration with
torch.utils.data.DataLoader - Directory-based loading with automatic labeling
- Plugin system for extending dataset types
- Designed with reproducibility and best practices in mind
Installation
Coming soon to PyPI!
For now, you can install the development version directly from source:
git clone https://github.com/yourusername/torch-datasets.git
cd torch-datasets
pip install -e .
🧑💻 Quick Start
from torchdatasets.vision.classification import FromSubDirDataset
from torch.utils.data import DataLoader
from torchvision import transforms
# Define any transformations
transform = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor(),
])
# Load images from a directory structure: root/class_x/xxx.png
dataset = ImageFolderDataset("path/to/images", transform=transform)
# Create a DataLoader
dataloader = DataLoader(dataset, batch_size=32, shuffle=True)
# Training loop
for images, labels in dataloader:
# Your model training code here
pass
🛠️ Project Status
| Feature | Status |
|---|---|
| Project initialized | ✅ Complete |
| Image datasets | 🚧 In Progress |
| Text datasets | ⏳ Planned |
| Audio datasets | ⏳ Planned |
| Tabular datasets | ⏳ Planned |
| Custom dataset interface | ⏳ Planned |
| Plugin support | ⏳ Planned |
| Benchmarking tools | ⏳ Planned |
Follow the repository for ongoing updates. Feature suggestions and pull requests are welcome!
🤝 Contributing
A full CONTRIBUTING.md is here.
📄 License
This project is licensed under the Apache License 2.0. See the full LICENSE file for details.
📬 Stay Connected
- 📘 PyTorch Documentation
- 🐞 Report Issues
- ⭐ Star the Repo to follow development
torch-datasets: Because data loading shouldn't slow you down.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file torch_custom_datasets-0.0.3.tar.gz.
File metadata
- Download URL: torch_custom_datasets-0.0.3.tar.gz
- Upload date:
- Size: 8.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.12.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
638ef24a74fa8b759c58df2a5b5623831406eef24910a32afbcbbcccae15ddaa
|
|
| MD5 |
7b5c79df92685a8545787501dd0a45d3
|
|
| BLAKE2b-256 |
6a1438e4761c942cf71e8b52f08ea6fbb417357a19df1137d65d467ded275107
|
Provenance
The following attestation bundles were made for torch_custom_datasets-0.0.3.tar.gz:
Publisher:
python-publish.yml on Shubh-Goyal-07/torch-datasets
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
torch_custom_datasets-0.0.3.tar.gz -
Subject digest:
638ef24a74fa8b759c58df2a5b5623831406eef24910a32afbcbbcccae15ddaa - Sigstore transparency entry: 352895632
- Sigstore integration time:
-
Permalink:
Shubh-Goyal-07/torch-datasets@40c6dab6f43df694b56a5880ba8494a49450d93a -
Branch / Tag:
refs/tags/v0.0.3 - Owner: https://github.com/Shubh-Goyal-07
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
python-publish.yml@40c6dab6f43df694b56a5880ba8494a49450d93a -
Trigger Event:
release
-
Statement type:
File details
Details for the file torch_custom_datasets-0.0.3-py3-none-any.whl.
File metadata
- Download URL: torch_custom_datasets-0.0.3-py3-none-any.whl
- Upload date:
- Size: 9.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.12.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
db09900e1138e4c04d484def4a2c636ce6790b83ee77bf574835b7375c7aa631
|
|
| MD5 |
f9ca80f82758a691029cbbdb06c0fe2a
|
|
| BLAKE2b-256 |
6affde72b0be23172f54bfb60169e497f38adc3f97b7dc1143bf2e9f509e6c3b
|
Provenance
The following attestation bundles were made for torch_custom_datasets-0.0.3-py3-none-any.whl:
Publisher:
python-publish.yml on Shubh-Goyal-07/torch-datasets
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
torch_custom_datasets-0.0.3-py3-none-any.whl -
Subject digest:
db09900e1138e4c04d484def4a2c636ce6790b83ee77bf574835b7375c7aa631 - Sigstore transparency entry: 352895646
- Sigstore integration time:
-
Permalink:
Shubh-Goyal-07/torch-datasets@40c6dab6f43df694b56a5880ba8494a49450d93a -
Branch / Tag:
refs/tags/v0.0.3 - Owner: https://github.com/Shubh-Goyal-07
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
python-publish.yml@40c6dab6f43df694b56a5880ba8494a49450d93a -
Trigger Event:
release
-
Statement type: