Skip to main content

LolaML - track your ML experiments

Project description

LolaML - track your ML experiments

Documentation Status Pipeline Status Coverage Report PyPI Version

Track your machine learning experiments with LolaML, never lose any information or forget which parameter yielded which results. Lola creates a simple JSON representation of the run that contains all the information you logged. The JSON can easily be shared to collaborate with friends and colleagues. Lola strives to be non-magic and simple but configurable.

Features:

  • a simple logging interface

  • a simple JSON representation of the logged data

  • works with any machine learning library

  • automatically creates an artifact folder for each run

  • automatically uploads artifacts to a remote bucket (if you want)

  • simple jupyter notebook dashboard (more to come…)

import lolaml as lola

# Use the run context manager to start/end a run
with lola.Run(project="mnist", prefix_path="data/experiments") as run:
    # a unique id for the run
    print(run.run_id)
    # store all artifacts (model files, images, etc.) here
    print(run.path)  # -> data/experiments/mnist/<run_id>

    run.log_param("lr", 0.1)
    run.log_param("epochs", 10)

    run.log_tags("WIP", "RNN")

    # Create and train your model...

    run.log_metric("loss", loss, step=1)
    run.log_metric("train_acc", train_acc, step=1)
    run.log_metric("val_acc", val_acc, step=1)

    model.save(os.path.join(run.path, "model.pkl"))

# After a run there is a lola_run*.json file under run.path.
# It contails all the information you logged.

After the run there is a JSON file that looks something like this:

{
    "project": "mnist",
    "run_id": "9a531da0-dc43-4dcc-8968-77fd480ff7ee",
    "status": "done",
    "path": "data/experiments/mnist/9a531da0-dc43-4dcc-8968-77fd480ff7ee",
    "run_file": "data/experiments/mnist/9a531da0-dc43-4dcc-8968-77fd480ff7ee/lola_run.json",
    "user": "stefan",
    "start_time": "2019-02-16 12:49:32.782958",
    "end_time": "2019-02-16 12:49:32.814529",
    "metrics": [
        {
            "name": "loss",
            "value": 1.5
            "step": 1,
            "ts": "2019-02-16 12:49:32.813750"
        },
        ...
    ],
    "params": {
        "lr": "0.1",
        "epochs": 10,
    },
    "tags": ["WIP", "RNN"],
    "artifacts": {
        "data/experiments/mnist/9a531da0-dc43-4dcc-8968-77fd480ff7ee/lola_run_9a531da0-dc43-4dcc-8968-77fd480ff7ee.json": {},
        ...
    },
    "git": {
        "sha": "41cb4fb11b7e937c602c2282b9275200c88b8797",
        "status": "...",
        "diff": "...",
    },
    "call_info": {
        "cwd": "some/where",
        "__file__": "somefile.py",
        "argv": [],
    }
}

Lola can automatically upload all artifacts to a remote storage bucket for you:

with lola.run(
    remote_location="gs://somewhere",
    remote_credentials="service_account.json",
) as run:
    # train and log ...

# All artifacts are uploaded now

The remote location can also be configured with the .lola.toml file.

Additionally, Lola offers some helpers to analyse the your experiments:

TODO add image of dashboard

Setup

Requirements

  • Python 3.6+

Installation

Install this library directly into an activated virtual environment:

$ pip install lolaml

or add it to your Poetry project:

$ poetry add lolaml

Misc

This project was generated with cookiecutter using jacebrowning/template-python. Thanks!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lolaml-0.0.4.tar.gz (14.1 kB view details)

Uploaded Source

Built Distribution

lolaml-0.0.4-py3-none-any.whl (31.7 kB view details)

Uploaded Python 3

File details

Details for the file lolaml-0.0.4.tar.gz.

File metadata

  • Download URL: lolaml-0.0.4.tar.gz
  • Upload date:
  • Size: 14.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/0.12.11 CPython/3.6.7 Linux/4.15.0-45-generic

File hashes

Hashes for lolaml-0.0.4.tar.gz
Algorithm Hash digest
SHA256 cbeedad5723ad3e0b6f8c0f3619819b2ce7502d77bf545fcc5c4f4bd6555de66
MD5 eaac34724bba9827978af86d16176e38
BLAKE2b-256 48abb6647bd63f8f6f9d28a38f24e150d355c247c9a5504789a444d1ca3d2dcb

See more details on using hashes here.

File details

Details for the file lolaml-0.0.4-py3-none-any.whl.

File metadata

  • Download URL: lolaml-0.0.4-py3-none-any.whl
  • Upload date:
  • Size: 31.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/0.12.11 CPython/3.6.7 Linux/4.15.0-45-generic

File hashes

Hashes for lolaml-0.0.4-py3-none-any.whl
Algorithm Hash digest
SHA256 95632de664773cb71f424d8488d095d3beb266837d6db68725e8044ca96722ae
MD5 86a4176524a1d5d8e9db8d6b253003de
BLAKE2b-256 9befd9061cc04bdddf84e697260af3354e48f5b392d6192f981eac8052ca2500

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page