paramsurvey 0.2.3

A toolkit for doing parameter surveys

paramsurvey

paramsurvey is a set of tools for creating and executing parameter surveys.

paramsurvey has a pluggable parallel backend. The supported backends at present are python's multiprocessing module, and computing cluster software ray. An mpi backend is planned.

Example

import time
import paramsurvey


def sleep_worker(pset, system_kwargs, user_kwargs, raw_stats):
    time.sleep(pset['duration'])
    return {'slept': pset['duration']}


paramsurvey.init(backend='multiprocessing')  # or 'ray', if you installed it

psets = [{'duration': 0.3}] * 5

results = paramsurvey.map(sleep_worker, psets, verbose=2)

for r in results.results:
    print(r['pset'], r['result'])

prints, in addition to some debugging output, a result from each of the 5 sleep_worker calls.

Philosophy

A parameter survey runs begins by initializing the software, specifying which backend ('multiprocessing' or 'ray').

A list of parameter sets (psets) is constructed.

The pararamsurvey.map() function executes the worker function once for each pset.

It returns a MapResults object, containing the results, performance statistics, and information about any failures.

Worker function limitations

The worker function runs in a different address space and possibly on a different server. It shouldn't access any global variables.

For hard-to-explain Python issues, be sure to define the worker function before calling paramsurvey.init(). The worker function should not be nested inside another function. On Windows, the main program file should have a if __name == '__main__' guard similar to the examples at the top of the Python multprocessing documentation.

The MapResults object

The MapResults object has several properties:

• results is a list of dictionaries; 'return' is the return value of the worker function, and 'pset' is the pset.
• failed is a list of failed psets, plus an extra '_exception' key if an exception was raised in the worker
• progress is a MapProgress object containing the details of pset execution: total, started, finished, failures, exceptions
• stats is a PerfStats object containing performance statistics

Installing

$ pip install paramsurvey
$ pip install paramsurvey[ray]