Search space optimization via gradient boosting regression
Project description
SpaceOpt: optimize discrete search space via gradient boosting regression
SpaceOpt is an optimization algorithm for discrete search spaces that uses gradient boosting regression to find the most promising candidates for evaluation by predicting their evaluation score. Training data is gathered sequentially and random or human-guided exploration can be easily incorporated at any stage.
Usage
If you have discrete search space, for example:
search_space = {
'a': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], # list of ordered numbers: ints
'b': [-4.4, -2.5, -1.5, 0.0, 3.7], # list of ordered numbers: floats
'c': [128, 256, 512, 1024], # another list of ordered numbers
'd': ['typeX', 'typeY', 'typeZ'], # categorical variable
'e': [True, False], # boolean variable
# ... (add as many as you need)
}
and if you can evaluate points from it:
spoint = {'a': 4, 'b': 0.0, 'c': 512, 'd': 'typeZ', 'e': False}
y = evaluation_function(spoint)
print(y) # 0.123456
and if you want to find points that maximize or minimize your evaluation function, in a better way than random search, then use SpaceOpt:
from spaceopt import SpaceOpt
spaceopt = SpaceOpt(search_space=search_space,
target_name='y',
objective='min') # or 'max'
for iteration in range(200):
if iteration < 20:
spoint = spaceopt.get_random() # exploration
else:
spoint = spaceopt.fit_predict() # exploitation
spoint['y'] = evaluation_function(spoint)
spaceopt.append_evaluated_spoint(spoint)
More examples here.
Installation
$ pip install spaceopt
Advanced
- get multiple points by setting
num_spoints
:
spoint_list = spaceopt.get_random(num_spoints=2)
# or
spoint_list = spaceopt.fit_predict(num_spoints=5)
- control exploitation behaviour by adjusting
sample_size
(default=10000), which is the number of candidates sampled for ranking (decreasingsample_size
increses exploration):
spoint = spaceopt.fit_predict(sample_size=100)
- add your own evaluation points to SpaceOpt:
my_spoint = {'a': 8, 'b': -4.4, 'c': 256, 'd': 'typeY', 'e': False}
my_spoint['y'] = evaluation_function(my_spoint)
spaceopt.append_evaluated_spoint(my_spoint)
-
be creative about how to use SpaceOpt;
-
learn more by reading the code, there are only 3 classes: SpaceOpt, Space and Variable.
License
MIT License (see LICENSE).
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.