Skip to main content

Efficiently Extract, Transform, and Load your dataset into PyTorch models

Project description

TorchETL

If you're working on classification problem, with dataset that is available in their native format (jpg, bmp, etc) and have PyTorch in your arsenal, you'll most likely feel that the DatasetFolder or ImageFolder is not good enough. So does vanilla torch.utils.data.Dataset. This library attempts to bridge that gap to effectively Extract, Transform, and Load your data by extending torch.utils.data.Dataset.

Main Features

Extract class would partition your dataset into train, validation, and test csv

TransformAndLoad class would Transform and consume your dataset efficiently

Prerequisites

Python 3.7.2 (other versions might work if type checking is supported)

torch

torchvision

numpy

pandas

opencv-python

sklearn

Or simply download requirements.txt and fire 'pip3 install -r requirements.txt'

Installing

pip3 install torchetl

Tutorial

See tutorial/Tutorial.ipynb

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

torchetl-0.3.9.tar.gz (4.8 kB view details)

Uploaded Source

Built Distribution

torchetl-0.3.9-py3-none-any.whl (8.6 kB view details)

Uploaded Python 3

File details

Details for the file torchetl-0.3.9.tar.gz.

File metadata

  • Download URL: torchetl-0.3.9.tar.gz
  • Upload date:
  • Size: 4.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.35.0 CPython/2.7.15+

File hashes

Hashes for torchetl-0.3.9.tar.gz
Algorithm Hash digest
SHA256 e20e6a470cb713dc320a4a4fa15b38020d4fedbfdb91424ae53a2833dc18559a
MD5 a0484c176a7ff7362214fbc9b8edca1b
BLAKE2b-256 8a70a96c6fbee3a56535cfb59954179a216361d07c268385af69c69a2920f3da

See more details on using hashes here.

File details

Details for the file torchetl-0.3.9-py3-none-any.whl.

File metadata

  • Download URL: torchetl-0.3.9-py3-none-any.whl
  • Upload date:
  • Size: 8.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.35.0 CPython/2.7.15+

File hashes

Hashes for torchetl-0.3.9-py3-none-any.whl
Algorithm Hash digest
SHA256 7ec4b274aba07652351949ee967a4b54296f2476075a5eb53fef861326e5ccf0
MD5 823e47502a3f98fafe2892bffe182d57
BLAKE2b-256 dd1851604485c0ad3744d372730f52444ac21fd0de2a02b3142b53fee8ce6cd8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page