LolaML - track your ML experiments

Track your machine learning experiments with LolaML, never lose any information or forget which parameter yielded which results. Lola creates a simple JSON representation of the run that contains all the information you logged. The JSON can easily be shared to collaborate with friends and colleagues. Lola strives to be non-magic and simple but configurable.

Features:

• a simple logging interface

• a simple JSON representation of the logged data

• works with any machine learning library

• automatically creates an artifact folder for each run

• automatically uploads artifacts to a remote bucket (if you want)

• simple jupyter notebook dashboard (more to come…)

import lolaml as lola

# Use the run context manager to start/end a run
with lola.Run(project="mnist", prefix_path="data/experiments") as run:
    # a unique id for the run
    print(run.run_id)
    # store all artifacts (model files, images, etc.) here
    print(run.path)  # -> data/experiments/mnist/<run_id>

    run.log_param("lr", 0.1)
    run.log_param("epochs", 10)

    run.log_tags("WIP", "RNN")

    # Create and train your model...

    run.log_metric("loss", loss, step=1)
    run.log_metric("train_acc", train_acc, step=1)
    run.log_metric("val_acc", val_acc, step=1)

    model.save(os.path.join(run.path, "model.pkl"))

# After a run there is a lola_run*.json file under run.path.
# It contails all the information you logged.

After the run there is a JSON file that looks something like this:

{
    "project": "mnist",
    "run_id": "9a531da0-dc43-4dcc-8968-77fd480ff7ee",
    "status": "done",
    "path": "data/experiments/mnist/9a531da0-dc43-4dcc-8968-77fd480ff7ee",
    "run_file": "data/experiments/mnist/9a531da0-dc43-4dcc-8968-77fd480ff7ee/lola_run.json",
    "user": "stefan",
    "start_time": "2019-02-16 12:49:32.782958",
    "end_time": "2019-02-16 12:49:32.814529",
    "metrics": [
        {
            "name": "loss",
            "value": 1.5
            "step": 1,
            "ts": "2019-02-16 12:49:32.813750"
        },
        ...
    ],
    "params": {
        "lr": "0.1",
        "epochs": 10,
    },
    "tags": ["WIP", "RNN"],
    "artifacts": {
        "data/experiments/mnist/9a531da0-dc43-4dcc-8968-77fd480ff7ee/lola_run_9a531da0-dc43-4dcc-8968-77fd480ff7ee.json": {},
        ...
    },
    "git": {
        "sha": "41cb4fb11b7e937c602c2282b9275200c88b8797",
        "status": "...",
        "diff": "...",
    },
    "call_info": {
        "cwd": "some/where",
        "__file__": "somefile.py",
        "argv": [],
    }
}

Lola can automatically upload all artifacts to a remote storage bucket for you:

with lola.run(
    remote_location="gs://somewhere",
    remote_credentials="service_account.json",
) as run:
    # train and log ...

# All artifacts are uploaded now

The remote location can also be configured with the .lola.toml file.

Additionally, Lola offers some helpers to analyse the your experiments:

TODO add image of dashboard

Setup

$ pip install lolaml

$ poetry add lolaml

Misc

This project was generated with cookiecutter using jacebrowning/template-python. Thanks!