modularyze 0.1.0

Installation

To install the latest version of modularyze, run this command in your terminal:

$ pip install modularyze

Example

The Modularize package exposes one central config-builder class called ConfBuilder. Using this class you can register arbitrary constructors and callables, render templated multi-file and dynamic configs, instantiate them and compare configs by hash or their normalized form.

To use modularyze in a project simply import it, register any callables your config might be using and point it to your configuration file. From there you can simply call build to build the config.

A simple example where we instantiate a machine learning pipeline could look something like this:

# File: imagenet.yaml

{% set use_pretrained = use_pretrained | default(True) %}
{% set imagenet_root = imagenet_root | default('datasets/imagenet') %}

network: &network
    !torchvision.models.resnet18
    pretrained: {{ use_pretrained }}

val_transforms: &val_transforms
    !torchvision.transforms.Compose
    - !torchvision.transforms.Resize [256]
    - !torchvision.transforms.CenterCrop [224]
    - !torchvision.transforms.ToTensor

dataset: &dataset
    !torchvision.transforms.datasets.ImageNet
    args:
      - {{ imagenet_root }}
    kwargs:
      split: 'val'
      transforms: *val_transforms

import torchvision
from modularyze import ConfBuilder

builder = ConfBuilder()
builder.register_multi_constructors_from_modules(torchvision)
conf = builder.build('imagenet.yaml')

Now the conf object is a python dictionary containing a fully initialized model, dataset and validation transforms. What about if you want to change a parameter on the fly? Say the imagenet folder changes? Easy, simply pass in a context:

conf = builder.build('imagenet.yaml', context={"imagenet_root": "new/path/to/dataset"})

In this way ypu can easily parameterize you configuration files. The provided context is usually a dictionary but it can even be the path to a (non-parameterized/vanilla) YAML file.

What about if we have the configuration for a model trainer in a different file? Imagine the file trainer.yaml instantiates a neural network trainer instance, we can include it by adding the following line to the above config file:

{% include 'trainer.yaml' %}

There are many more neat things you can do when you combine the powers of YAML and Jinja, please refer to the documentation for more.