NoneChucks is a library for PyTorch that allows you to drop samples from your data and ensures that your Datasets and Transforms do not fail because of a few samples that are not parseable.
Project description
nonechucks
Introduction
The point of any data processing pipeline is to reduce the burden of doing messy and repetitive tasks for the programmer. PyTorch's flexibly designed data processing module makes it a breeze to do common tasks like shuffling, batching, and iterating through datasets. However, both the dataset and sampler expect you to rid your dataset of any unwanted or corrputed samples before you feed it to them, as they provide no easy way to define what action to take in case an invalid sample is encountered. In fact, for many applications, this defeats the whole point of using a data processing module.
Nonechucks allows you to wrap your existing datasets and samplers
Installation
To install Nonechucks, simply use pip:
$ pip install nonechucks
Usage
Examples
SafeDataset
SafeSampler
Contributing
We appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion. If you plan to contribute new features, utility functions or extensions, please first open an issue and discuss the feature with us.
Licensing
Nonechucks is MIT licensed.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.