Skip to main content

Pre-packages Pytorch-Lightning datasets

Project description

Alt text

LightningData Modules

Pre-packages Pytorch-Lightning datasets.

Installation

To install this library, simply run the following command:

pip install lightingdata_modules

Installing the LightningData Modules should automatically install suitable dependencies.

Customized Dataset Support

LightningData Modules downloads, installs and prepares customized Pytorch Lightning datamodules with just one line of code.

Example from Domain Adaptation (DomainNet):

# import the custom DomainNet module
import lightningdata_modules.domain_adaptation.domainNet_datamodule as domainNet

# initialize the data module with the "real" domain from DomainNet
my_data_module = domainNet.DomainNetDataModule(data_dir="./dataset/", domain="real")

The data module can now be passed to the Pytorch Lighting Trainer instance or be used as a standalone Dataloader:

# check if the dataset already exists in the data_dir, download domain-specific lmdb database from cloud storage otherwise
my_data_module.prepare_data()

# initialize the train and test set according to setup properties
my_data_module.setup()

# create the train DataLoader on-the-fly
train_loader = my_data_module.train_dataloader()

List of Available Datasets

Domain Adaptation Federated Learning Meta Learning
DomainNet EMNIST Omiglot
Office31 Mini Imagenet
OfficeHome
Digit-Five

Class Diagram

Alt text

Ack

Thanks to TorchMeta: https://github.com/tristandeleu/pytorch-meta

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lightningdata-modules-0.1.9.tar.gz (67.5 kB view details)

Uploaded Source

File details

Details for the file lightningdata-modules-0.1.9.tar.gz.

File metadata

  • Download URL: lightningdata-modules-0.1.9.tar.gz
  • Upload date:
  • Size: 67.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.8.12

File hashes

Hashes for lightningdata-modules-0.1.9.tar.gz
Algorithm Hash digest
SHA256 7aba3152672e9ddbcccc7e59130fe775e41fdb5a6cb629c0517f33e7da9a581e
MD5 eab40e071185058bf7f4a5e44d58be04
BLAKE2b-256 a26900617670585883040d483fc90870f70dcfae1847556e1f00be0a8db135ea

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page