A Python module for parallel optimization of expensive black-box functions
Project description
blackbox: A Python module for parallel optimization of expensive black-box functions
What is this?
A minimalistic and easy-to-use Python module that efficiently searches for a global minimum of an expensive black-box function (e.g. optimal hyperparameters of simulation, neural network or anything that takes significant time to run). User needs to provide a function, a search domain (ranges of each input parameter) and a total number of function calls available. A code scales well on multicore CPUs and clusters: all function calls are divided into batches and each batch is evaluated in parallel.
A mathematical method behind the code is described in this arXiv note (there were few updates to the method recently): https://arxiv.org/pdf/1605.00998.pdf
Don't forget to cite this note if you are using method/code.
Demo
(a) - demo function (unknown to a method).
(b) - running a procedure using 15 evaluations.
(c) - running a procedure using 30 evaluations.
How do I represent my objective function?
It simply needs to be wrapped into a Python function. An external application, if any, can be accessed using system call.
def fun(par):
...
return output
par
is a vector of input parameters (a Python list), output
is a scalar value to be minimized.
How do I run the procedure?
Just place blackbox.py
into your working directory. Main file should look like this:
import blackbox as bb
def fun(par):
return par[0]**2 + par[1]**2 # dummy example
best_params = bb.search_min(f = fun, # given function
domain = [ # ranges of each parameter
[-10., 10.],
[-10., 10.]
],
budget = 40, # total number of function calls available
batch = 4, # number of calls that will be evaluated in parallel
resfile = 'output.csv') # text file where results will be saved
Important:
- All function calls are divided into batches and each batch is evaluated in parallel. Total number of batches is
budget/batch
. The value ofbatch
should correspond to the number of available computational units. - An optional parameter
executor = ...
should be specified withinbb.search_min()
in case when custom parallel engine is used (ipyparallel, dask.distributed, pathos etc).executor
should be an object that has amap
method.
How about results?
In addition to search_min()
returning list of optimal parameters, all trials are sorted by function value (best ones at the top) and saved in a text file with the following structure:
Parameter #1 | Parameter #2 | ... | Parameter #n | Function value |
---|---|---|---|---|
+1.6355e+01 | -4.7364e+03 | ... | +6.4012e+00 | +1.1937e-04 |
... | ... | ... | ... | ... |
Author
Paul Knysh (paul.knysh@gmail.com)
I receive tons of useful feedback that helps me to improve the code. Feel free to email me if you have any questions or comments.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for black_box-1.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2ece8ff9aebd0e485045ad166c844adc1908c558bab8418bcaa5c38ce8caca05 |
|
MD5 | f93e913f69244e1a43a738e4400bb4c8 |
|
BLAKE2b-256 | fe4b868cfb7e103f9d12508ded27f172bb122376054487c5c1213c086c973585 |