Skip to main content

LolaML - track your ML experiments

Project description

LolaML - track your ML experiments

Documentation Status Pipeline Status Coverage Report PyPI Version

Track your machine learning experiments with LolaML, never lose any information or forget which parameter yielded which results. Lola creates a simple JSON representation of the run that contains all the information you logged. The JSON can easily be shared to collaborate with friends and colleagues. Lola strives to be non-magic and simple but configurable.

Features:

  • a simple logging interface

  • a simple representation of the logged data

  • works with any machine learning library

  • automatically creates an artifact folder for each run

  • automatically uploads artifacts to a remote bucket (if you want)

  • simple jupyter notebook dashboard (more to come…)

import lolaml as lola

# Use the run context manager to start/end a run
with lola.Run(project="mnist", prefix_path="data/experiments") as run:
    # a unique id for the run
    print(run.run_id)
    # store all artifacts (model files, images, etc.) here
    print(run.path)  # -> data/experiments/<run_id>

    run.log_param("lr", 0.1)
    run.log_param("epochs", 10)

    run.log_tags("WIP", "RNN")

    # Create and train your model...

    run.log_metric("loss", loss, step=1)
    run.log_metric("train_acc", train_acc, step=1)
    run.log_metric("val_acc", val_acc, step=1)

    model.save(os.path.join(run.path, "model.pkl"))

# After a run there is a lola_run*.json file under run.path.
# It contails all the information you logged.

After the run there is a JSON file that looks something like this:

{
    "project": "mnist",
    "run_id": "9a531da0-dc43-4dcc-8968-77fd480ff7ee",
    "status": "done",
    "path": "data/experiments/9a531da0-dc43-4dcc-8968-77fd480ff7ee",
    "user": "stefan",
    "start_time": "2019-02-16 12:49:32.782958",
    "end_time": "2019-02-16 12:49:32.814529",
    "metrics": [
        {
            "name": "loss",
            "value": 1.5
            "step": 1,
            "ts": "2019-02-16 12:49:32.813750"
        },
        ...
    ],
    "params": {
        "lr": "0.1",
        "epochs": 10,
    },
    "tags": ["WIP", "RNN"],
    "artifacts": {
        "data/experiments/9a531da0-dc43-4dcc-8968-77fd480ff7ee/lola_run_9a531da0-dc43-4dcc-8968-77fd480ff7ee.json": {},
        ...
    },
    "git": {
        "sha": "41cb4fb11b7e937c602c2282b9275200c88b8797",
        "status": "...",
        "diff": "...",
    },
    "call_info": {
        "__file__": "somefile.py",
        "argv": [],
    }
}

Lola can automatically upload all artifacts to a remote storage bucket for you:

with lola.run(
    remote_location="gs://somewhere",
    remote_credentials="service_account.json",
) as run:
    # train and log ...

# All artifacts are uploaded now

The remote location can also be configured with the .lola.toml file.

Additionally, Lola offers some helpers to analyse the your experiments:

TODO add image of dashboard

Setup

Requirements

  • Python 3.6+

Installation

Install this library directly into an activated virtual environment:

$ pip install lolaml

or add it to your Poetry project:

$ poetry add lolaml

Misc

This project was generated with cookiecutter using jacebrowning/template-python. Thanks!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lolaml-0.0.3.tar.gz (14.1 kB view hashes)

Uploaded Source

Built Distribution

lolaml-0.0.3-py3-none-any.whl (31.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page