Skip to main content

Hyperparameter optimization via gradient boosting regression

# SpaceOpt: hyperparameter optimization via gradient boosting regression

SpaceOpt is a hyperparameter optimization algorithm that uses gradient boosting regression to find the most promising candidates for the next trial by predicting their evaluation score.

## Installation

\$ pip install spaceopt

## Usage

1. Define a discrete hyperparameter search space, for example:
search_space = {
'a': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],  # list of ordered numbers: ints
'b': [-3.5, -0.1, 0.0, 2.5, 10.0],    # list of ordered numbers: floats
'c': [256, 512, 1024, 2048],          # another list of ordered numbers
'd': ['ABC', 'IJK', 'XYZ'],           # categorical variable
'e': [True, False],                   # boolean variable
# ... (add as many as you need)
}
1. Define your evaluation function:
def evaluation_function(spoint: dict) -> float:
# your code (e.g. model fit)
return y  # score (e.g. model accuracy)

spoint = {'a': 4, 'b': 0.0, 'c': 512, 'd': 'XYZ', 'e': False}
y = evaluation_function(spoint)
1. Use SpaceOpt for a hyperparameter optimization:
from spaceopt import SpaceOpt

spaceopt = SpaceOpt(search_space=search_space,
target_name='y',
objective='maximize')  # or 'minimize'

for iteration in range(200):
if iteration < 20:
spoint = spaceopt.get_random()     # exploration
else:
spoint = spaceopt.fit_predict()    # exploitation

spoint['y'] = evaluation_function(spoint)
spaceopt.append_evaluated_spoint(spoint)

More examples here.

## Advanced

• get multiple points by setting num_spoints:
spoints = spaceopt.get_random(num_spoints=2)
# or
spoints = spaceopt.fit_predict(num_spoints=5)
• control exploration vs. exploitation behaviour by adjusting sample_size (default=10000), which is the number of candidates sampled for ranking:
spoint = spaceopt.fit_predict(sample_size=1000)  # decreasing sample_size increses exploration
spoint = spaceopt.fit_predict(sample_size=100000)  # increasing sample_size increses exploitation
• add manually selected evaluation points to SpaceOpt:
my_spoint = {'a': 8, 'b': -3.5, 'c': 256, 'd': 'IJK', 'e': False}
my_spoint['y'] = evaluation_function(my_spoint)
spaceopt.append_evaluated_spoint(my_spoint)

## License

MIT License (see LICENSE).

## Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

### Source Distribution

spaceopt-0.2.2.tar.gz (7.7 kB view hashes)

Uploaded Source

### Built Distribution

spaceopt-0.2.2-py3-none-any.whl (7.8 kB view hashes)

Uploaded Python 3