experiment-collection 0.1.5

Experiment collection

Experiment collection

A set of utilities for storing and organizing experiments

Install

pip install experiment-collection

You can find another versions at releases or pypi.

Usage_example

Experiment - structure with experiment data;

ExperimentCollectionLocal - local experiment storage;

ExperimentCollectionRemote - remote experiment storage; could combine data from multiply sources.

from experiment_collection.experiments import Experiment, ExperimentCollectionLocal, ExperimentCollectionRemote

exps1 = ExperimentCollectionLocal('exp_test.db')
exps2 = ExperimentCollectionRemote('http://127.0.0.1/', 'exp_test')

for i in range(10):
    exp = Experiment('name_{}'.format(i))
    exp.set_metrics({'lr': 0.1})
    exp.set_params({'auc': 0.5})

    exps1.add_experiment(exp)
    exps2.add_experiment(exp)

    assert exps1.check_experiment(exp)
    assert exps2.check_experiment(exp)

# Delete latest experiment
exps1.delete_experiment(exp)
exps2.delete_experiment(exp)

assert not exps1.check_experiment(exp)
assert not exps2.check_experiment(exp)

View all results. All metrics and params auto flatten into single columns.

exps1.get_experiments()

     name                       time  params_auc  metrics_lr
0  name_0 2020-09-30 23:56:39.932871         0.5         0.1
1  name_1 2020-09-30 23:56:40.216424         0.5         0.1
2  name_2 2020-09-30 23:56:40.620029         0.5         0.1
3  name_3 2020-09-30 23:56:40.916781         0.5         0.1
4  name_4 2020-09-30 23:56:41.240535         0.5         0.1
5  name_5 2020-09-30 23:56:41.567865         0.5         0.1
6  name_6 2020-09-30 23:56:41.861890         0.5         0.1
7  name_7 2020-09-30 23:56:42.177155         0.5         0.1
8  name_8 2020-09-30 23:56:42.507883         0.5         0.1
9  name_9 2020-09-30 23:56:42.818714         0.5         0.1

License

MIT License

Copyright (c) 2020 AsciiShell (Aleksey Podchezertsev)