Skip to main content

Efficiently Extract, Transform, and Load your dataset into PyTorch models

Project description

TorchETL

If you're working on classification problem, with dataset that is available in their native format (jpg, bmp, etc) and have PyTorch in your arsenal, you'll most likely feel that the DatasetFolder or ImageFolder is not good enough. So does vanilla torch.utils.data.Dataset. This library attempts to bridge that gap to effectively Extract, Transform, and Load your data by extending torch.utils.data.Dataset.

Main Features

Extract class would partition your dataset into train, validation, and test csv

TransformAndLoad class would Transform and consume your dataset efficiently

Prerequisites

Python 3.7.2 (other versions might work if type checking is supported)

torch

torchvision

numpy

pandas

opencv-python

sklearn

Or simply download requirements.txt and fire 'pip3 install -r requirements.txt'

Installing

pip3 install torchetl

Tutorial

See tutorial/Tutorial.ipynb

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

torchetl-0.3.9.tar.gz (4.8 kB view hashes)

Uploaded source

Built Distribution

torchetl-0.3.9-py3-none-any.whl (8.6 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page