args-to-db 0.0.6

Runs python script in specified argument combinations and produces a pandas dataframe of all results.

args_to_db - An Argument Optimisation Data Generation Tool

    pip install args_to_db

You want to analyze a (python) script for different arguments/settings? - Argument optimization is becoming more and more important in many application areas. args_to_db is an attempt to generalize and simplify the process of running a programm in different modes or configurations and combining the resulting datasets to allow for further analysis.

When should I use args_to_db?

You have a programm which is very general in its application, for example a solver framework for linear system of equations. This could look something like

    python solver.py A b [--solver SOLVER] [--preconditioner PRECONDITIONER]

Different inputs vary performance of solving methods dramatically. So we want to optimize the solver and preonditiner used for a specific linear system of equations.

How do I use args_to_db?

So we want to compare different argument configurations. args_to_db takes them in the array form:

    argument_configurations = [['--solver', 'cg'],
                               ['--solver', 'cg', '--preconditioner', 'jacobi'],
                               ['--solver', 'lu']]

And then we run all cases via:

    args_to_db.run('solver.py', argument_configurations, threads=3)

This spawns upto 3 concurrent processes each running the specified command. As complex applications are mostly reliant on argument optimisations, parallel execution is supported intuitively.

Data collection

We want to run the test cases individually and not inside a bigger (benchmarking) framework, but spawning different process yields the problem of data collection. Therfore the scripts internally log the results and args_to_db handles combination of those later on.

    args_to_db.add_result('solver_time', 20.3)
    args_to_db.add_result('solver_state', 'converged')

    args_to_db.write_results(__file__, args) # args currently is an argparse object

This results in single file outputs of each script. Which are then combined into a single pandas dataframe after all simulations are completed.