From PyPi

Installing dbcollection using pip is simple. For that purpose, simply do the following command:

$ pip install dbcollection

From source

To install dbcollection from source you need to do the following setps:

• Clone the repo to your hard drive:

$ git clone --recursive https://github.com/dbcollection/dbcollection

• cd to the dbcollection folder and do the command

$ python setup.py install

Basic usage

Using the module is pretty straight-forward. To import it just do:

>>> import dbcollection as dbc

To load a dataset, you only need to use a single method that returns a data loader object which can then be used to fetch data from.

>>> mnist = dbc.load('mnist')

This data loader object contains information about the dataset’s name, task, data, cache paths, set splits, and some methods for querying and loading data from the HDF5 metadata file.

For example, if you want to know how the data is structured inside the metadata file, you can simply do the following:

>>> mnist.info()

> Set: test
   - classes,        shape = (10, 2),          dtype = uint8
   - images,         shape = (10000, 28, 28),  dtype = uint8,  (in 'object_ids', position = 0)
   - labels,         shape = (10000,),         dtype = uint8,  (in 'object_ids', position = 1)
   - object_fields,  shape = (2, 7),           dtype = uint8
   - object_ids,     shape = (10000, 2),       dtype = uint8

   (Pre-ordered lists)
   - list_images_per_class,  shape = (10, 1135),  dtype = int32

> Set: train
   - classes,        shape = (10, 2),          dtype = uint8
   - images,         shape = (60000, 28, 28),  dtype = uint8,  (in 'object_ids', position = 0)
   - labels,         shape = (60000,),         dtype = uint8,  (in 'object_ids', position = 1)
   - object_fields,  shape = (2, 7),           dtype = uint8
   - object_ids,     shape = (60000, 2),       dtype = uint8

   (Pre-ordered lists)
   - list_images_per_class,  shape = (10, 6742),  dtype = int32

To fetch data samples from a field, its is as easy as calling a method with the set and field names and the row id(s) you want to select. For example, to retrieve the 10 first images all you need to do is the following:

>>> imgs = mnist.get('train', 'images', range(10))
>>> imgs.shape
(10, 28, 28)

Note: For more information about using this module, please check the documentation or the available notebooks for guidance.

Notebooks

For a more pratical introduction to dbcollection’s module for managing datasets and fetching data, there are some python notebooks available in the notebooks/ folder for a more hands-on tutorial on how to use this package.