Parametrise (python3 only) and execute Jupyter notebooks
Project description
run_jnb
run_jnb is a python package and command line tool for parametrising (python3 only) and executing Jupyter notebooks.
- Source: https://github.com/hz-inova/run_jnb
- Platform: Independent
- Development Status: Alpha
Installion
pip install run_jnb
Short Description
For a notebook written in python one can find the possible parameters. This is achieved by parsing the abstract syntax tree of the code cells. A variable can be a possible parameter if:
- it is defined in a cell that contains only comments or assignments,
- its name is not used as a global variable in the current cell (beside the assignment) nor previously.
One can pass arguments as keyword arguments or in a json format (file or string). For safety reasons, in order to avoid any code injection, only json serializable keywords arguments are available. The keyword arguments are firstly encoded in json format using the standard json encoder. The json content is decoded into python objects using the standard json decoder and it is mapped to a variable assignment by unpacking it. The assignments are appended at the end of the cell where they are initially defined.
The generated notebook (parametrised or not) can be easily executed (the implementation relies on nbconvert Executing notebooks).
The package contains two public functions possible_parameter and run_jnb (see the docstring).
>>> from run_jnb import possible_parameter, run_jnb
run_jnb can be used also as a command line tool and its documentation is available via
run_jnb -h
Simple Example
Consider the notebook.
possible_parameter returns a list of possible parameters with their name, value and cell index. The list is sorted alphabetically by the name of the possible parameters.
>>> possible_parameter('./Power_function.ipynb')
[PossibleParameter(name='exponent', value=2, cell_index=7),
PossibleParameter(name='np_arange_args', value={'start': -10, 'stop': 10, 'step': 0.01}, cell_index=4)]
The notebook contains several possible parameters.
Next we will parametrise the exponent using run_jnb.
run_jnb returns a tuple (output absolute path, error prompt number, error type, error value, error traceback). One can easily parametrise and execute a notebook
# Parametrise the noteboook and not execute the notebook
>>> run_jnb('./Power_function.ipynb', return_mode='parametrised_only', exponent=1)
# Parametrise and execute the notebook
>>> run_jnb('./Power_function.ipynb', return_mode=True, exponent=1)
('.../_run_jnb/Power_function-output.ipynb', None, None, None, None)
Please see the exported notebook by only parametrising and by parametrising and executing the initial notebook. Same output can be obtained by using arg parameter:
>>> run_jnb('.../Power_function.ipynb', return_mode=True, arg='{"exponent":1}')
or using the command line tool:
# " can be escaped by \"
$ run_jnb ./Power_function.ipynb -m true -a "{\"exponent\":1}" -vvv
".../_run_jnb/Power_function-output.ipynb",,,,
At command line the output is returned only in verbose mode (the tuple is serialised as a csv).
np_arange_args and exponent can be parametrised:
# parametrise using keyword arguments
>>> run_jnb('./Power_function.ipynb', return_mode=True, exponent=3, np_arange_args={'start':-20,'stop':20,'step':0.1})
# parametrise mixing keyword arguments and arg parameter
>>> run_jnb('./Power_function.ipynb', return_mode=True, arg='{"exponent":3}', np_arange_args={'start':-20,'stop':20,'step':0.1})
# parametrise using arg parameter with a json file
>>> run_jnb('./Power_function.ipynb', return_mode=True, arg='./power_function_arg.json')
('.../_run_jnb/Power_function-output (1).ipynb', None, None, None, None)
where in the last example power_function_arg.json contains:
{
"exponent": 3,
"np_arange_args": {
"start": -20,
"stop": 20,
"step": 0.1
}
}
Please see the generated notebook.
If the generated notebook contains an error:
>>> run_jnb('./Power_function.ipynb', return_mode=True, exponent=1, np_arange_args={'step':0.1})
('.../_run_jnb/Power_function-output (2).ipynb', 3, 'TypeError', "Required argument 'start' (pos 1) not found", ...)
the second element in the returned tuple (3 in thise case) is the prompt number of the cell where the error was caught (please see the generated notebook).
Dependencies
License
Acknowledgments
nbrun and nbparameterise were a source of inspiration.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.