Skip to main content

A DataLoader library for Continual Learning in PyTorch.

Project description

Continual Loader (CLLoader)

A library for PyTorch's loading of datasets in the field of Continual Learning

Aka Continual Learning, Lifelong-Learning, Incremental Learning, etc.


from import DataLoader

from clloader import CLLoader
from clloader.datasets import MNIST

clloader = CLLoader(
    MNIST("my/data/path", download=True),

print(f"Number of classes: {clloader.nb_classes}.")
print(f"Number of tasks: {clloader.nb_tasks}.")

for task_id, (train_dataset, test_dataset) in enumerate(clloader):
    train_loader = DataLoader(train_dataset)
    test_loader = DataLoader(test_dataset)

    # Do your cool stuff here

Supported Scenarios

Name Acronym  Supported
New Instances  NI :x:
New Classes  NC :white_check_mark:
New Instances & Classes  NIC :x:

Supported Datasets:

Note that the task sizes are fully customizable.

Name Nb classes  Image Size Automatic Download
MNIST 10  28x28x1 :white_check_mark:
Fashion MNIST 10  28x28x1 :white_check_mark:
KMNIST 10  28x28x1 :white_check_mark:
EMNIST 10  28x28x1 :white_check_mark:
QMNIST 10  28x28x1 :white_check_mark:
MNIST Fellowship 30  28x28x1 :white_check_mark:
CIFAR10 10 32x32x3 :white_check_mark:
CIFAR100 100 32x32x3 :white_check_mark:
CIFAR Fellowship 110 32x32x3 :white_check_mark:
ImageNet100 100 224x224x3 :x:
ImageNet1000 1000 224x224x3 :x:
Permuted MNIST 10 + X * 10 224x224x3 :white_check_mark:

Furthermore some "Meta"-datasets are available:

  • InMemoryDataset: for in-memory numpy array
  • PyTorchDataset: for any dataset defined in torchvision
  • ImageFolderDataset: for datasets having a tree-like structure, with one folder per class
  • Fellowship: to combine several datasets

Sample Images


Task 0 Task 1 Task 2 Task 3 Task 4


Task 0 Task 1 Task 2 Task 3 Task 4


Task 0 Task 1 Task 2 Task 3 Task 4

MNIST Fellowship (MNIST + FashionMNIST + KMNIST):

Task 0 Task 1 Task 2


Task 0 Task 1 Task 2 Task 3 Task 4

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

clloader-arthurdouillard-0.0.1.tar.gz (409.8 kB view hashes)

Uploaded Source

Built Distribution

clloader_arthurdouillard-0.0.1-py3-none-any.whl (11.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page