Scientific Experiment Exploration framework
SciExp² (aka SciExp square or simply SciExp2) stands for Scientific Experiment Exploration, which provides a framework for easing the workflow of creating, executing and evaluating experiments.
The driving idea of SciExp² is that of quick and effortless design-space exploration. It is divided into the following main pieces:
- Launchgen: Aids in defining experiments as a permutation of different parameters in the design space. It creates the necessary files to run these experiments (configuration files, scripts, etc.), which you define as templates that get substituted with the specific parameter values of each experiment.
- Launcher: Takes the files of launchgen and runs these experiments on different execution platforms like regular local scripts or cluster jobs. It takes care of tracking their correct execution, and allows selecting which experiments to run (e.g., those with specific parameter values, or those that were not successfully run yet).
- Data: Aids in the process of collecting and analyzing the results of the experiments. Results are automatically collected into a data structure that maintains the relationship between each result and the parameters of the experiment that produced it. With this you can effortlessly perform complex tasks such as inspecting the results or calculating statistics of experiment sub-sets, based on their parameter values.