Move-carrots

Simple helper for reproducible (research) experiments.

Goals

This package aims to make one's life easier with machine learning experiments. To achieve this goal, this package basically does the following two things.

1. Run experiments with some config files.

• No need for long commands.
• Reliable and elegant than notebooks.

2. Built-in support to record, view, and reproduce your experiments.

:P)

Quick start

WIP.

Core features

1. Define your experiment in a human friendly YAML file: main function, parameters, data files, logging, etc.
2. Codes will be automatically checked in to git with proper tags. Datas will be also version-controlled in a proper manner.
3. Each experiment will be dumped into a database, including the config, the logging, and the versions of code and data.

Design

Config file

The config file includes:

• PYTHONPATH
• main function and parameters
• misc
  • experiment name
  • logging

Version control

The goal is to re-run one experiment within one command. To achieve this, we will have both codes and data versioned in a proper way.

Codes All local changes, if there are any, will be committed into Git before a config runs. Commits information will be stored in log files and the head commit will be tagged as f"{name}@{time}" for future reference.

Data Data will be stored in the database, which will be keyed by name and md5 value.

Database

The database has two parts: experiment logging and version control.

1. Experiment logging shows what experiments have been conducted, what's the input/output of one experiment. Based on it, more analysis can be elaborated.
2. Version control is responsible for recording all versions of codes (through Git) and data (local storage).