Skip to main content

Framework for Reproducible ExperimenTs.

Project description

fret

Travis (.org) Coveralls github Documentation Status PyPI PyPI - Python Version

Framework for Reproducible ExperimenTs. Read on for a quick guide. Full documentation here.

Installation

From pip:

pip install fret

From source: clone the repository and then run: python setup.py install.

Tutorial

Basic Usage

Create a file named app.py with content:

import fret

@fret.command
def run(ws):
    model = ws.build()
    print(model)

@fret.configurable
class Model:
    def __init__(self, x=3, y=4):
        ...

Then under the same directory, you can run:

$ fret config Model
[ws/_default] configured "main" as "Model" with: x=3, y=4
$ fret run
Model(x=3, y=4)
$ fret config Model -x 5 -y 10
[ws/_default] configured "main" as "Model" with: x=5, y=10
$ fret run
Model(x=5, y=10)

Using Workspace

You can specify different configuration in different workspace:

$ fret -w ws/model1 config Model
[ws/model1] configured "main" as "Model" with: x=3, y=4
$ fret -w ws/model2 config Model -x 5 -y 10
[ws/model2] configured "main" as "Model" with: x=5, y=10
$ fret -w ws/model1 run
Model(x=3, y=4)
$ fret -w ws/model2 run
Model(x=5, y=10)

Save/Load

import fret

@fret.command
def train(ws):
    model = ws.build()
    model.train()
    ws.save(model, 'trained')

@fret.command
def test(ws):
    model = ws.load('ws/best/snapshot/main.trained.pt')
    print(model.weight)

@fret.configurable(states=['weight'])
class Model:
    def __init__(self):
        self.weight = 0
    def train(self):
        self.weight = 23
$ fret -w ws/best config Model
[ws/_default] configured "main" as "Model"
$ fret -w ws/best train
$ fret test
23

An Advanced Workflow

In app.py:

import time
import fret

@fret.configurable(states=['value'])
class Model:
    def __init__(self):
        self.value = 0

@fret.command
def resumable(ws):
    model = ws.build()
    with ws.run('exp-1') as run:
        run.register(model)
        cnt = run.acc()
        for e in fret.nonbreak(run.range(5)):
            # with `nonbreak`, the program always finish this loop before exit
            model.value += e
            time.sleep(0.5)
            cnt += 1
            print('current epoch: %d, sum: %d, cnt: %d' %
                  (e, model.value, cnt))

Then you can stop and restart this program anytime, with consistent results:

$ fret resumable
current epoch: 0, sum: 0, cnt: 1
current epoch: 1, sum: 1, cnt: 2
^CW SIGINT received. Delaying KeyboardInterrupt.
current epoch: 2, sum: 3, cnt: 3
Traceback (most recent call last):
    ...
KeyboardInterrupt
W cancelled by user
$ fret resumable
current epoch: 3, sum: 6, cnt: 4
current epoch: 4, sum: 10, cnt: 5

Submodule

@fret.configurable
class A:
    def __init__(self, foo):
        ...

@fret.configurable(submodules=['sub'], build_subs=False)
class B:
    def __init__(self, sub, bar=3):
        self.sub = sub(foo='bar')   # call sub to build submodule
$ fret config sub A
[ws/_default] configured "sub" as "A"
$ fret config B
[ws/_default] configured "main" as "B" with: sub='sub', bar=3
$ fret run
B(sub=A(), bar=3)

Inheritance

@fret.configurable
class A:
    def __init__(self, foo='bar', sth=3):
        ...

@fret.configurable
class B(A):
    def __init__(self, bar=3, **others):
        super().__init__(**others)
        ...
$ fret config B -foo baz -bar 0
[ws/_default] configured "main" as "B" with: bar=0, foo='baz', sth=3
$ fret run
B(bar=0, foo='baz', sth=3)

Internals

>>> config = fret.Configuration({'foo': 'bar'})
>>> config
foo='bar'

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for fret, version 0.2.7.post1
Filename, size File type Python version Upload date Hashes
Filename, size fret-0.2.7.post1-py3-none-any.whl (18.6 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size fret-0.2.7.post1.tar.gz (16.5 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page