fastai-datasets 0.0.5

Leveraging fastai to easily load and handle datasets

fastai-datasets

Docs

See https://irad-zehavi.github.io/fastai-datasets/

Install

pip install fastai_datasets

How to use

As an nbdev library, fatai_datasets supports import * (without importing unwanted symbols):

from fastai_datasets.all import *

Here are a few usage examles:

Easily load a dataset

mnist = MNIST()
mnist.dls().show_batch()

Show the class distribution

mnist.plot_class_distribution()

<div>
  <progress value='10' class='' max='10' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [10/10 00:00&lt;00:00 Class map: partitioning]
</div>

Sample a subset

Whole datasets:

mnist

[(#60000) [(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7))...]
(#10000) [(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7)),(PILImage mode=RGB size=28x28, TensorCategory(7))...]]

Subset:

mnist.random_sub_dsets(1000)

[(#865) [(PILImage mode=RGB size=28x28, TensorCategory(3)),(PILImage mode=RGB size=28x28, TensorCategory(1)),(PILImage mode=RGB size=28x28, TensorCategory(3)),(PILImage mode=RGB size=28x28, TensorCategory(0)),(PILImage mode=RGB size=28x28, TensorCategory(9)),(PILImage mode=RGB size=28x28, TensorCategory(8)),(PILImage mode=RGB size=28x28, TensorCategory(9)),(PILImage mode=RGB size=28x28, TensorCategory(1)),(PILImage mode=RGB size=28x28, TensorCategory(8)),(PILImage mode=RGB size=28x28, TensorCategory(1))...]
(#135) [(PILImage mode=RGB size=28x28, TensorCategory(3)),(PILImage mode=RGB size=28x28, TensorCategory(9)),(PILImage mode=RGB size=28x28, TensorCategory(4)),(PILImage mode=RGB size=28x28, TensorCategory(1)),(PILImage mode=RGB size=28x28, TensorCategory(4)),(PILImage mode=RGB size=28x28, TensorCategory(5)),(PILImage mode=RGB size=28x28, TensorCategory(0)),(PILImage mode=RGB size=28x28, TensorCategory(4)),(PILImage mode=RGB size=28x28, TensorCategory(1)),(PILImage mode=RGB size=28x28, TensorCategory(9))...]]

Construct a subset based on classes

cifar10 = CIFAR10()
dig_frog_bird = cifar10.by_target['dog'] + cifar10.by_target['frog'] + cifar10.by_target['bird']
dig_frog_bird.dls().show_batch()

<div>
  <progress value='10' class='' max='10' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [10/10 00:00&lt;00:00 Class map: partitioning]
</div>

Construct a dataset of similarity pairs

Pairs(cifar10, .01).dls().show_batch()

<div>
  <progress value='50' class='' max='50' style='width:300px; height:20px; vertical-align: middle;'></progress>
  100.00% [50/50 00:00&lt;00:00 Generating negative pairs]
</div>