hyperactive 2.0.0

A hyperparameter optimization toolbox for convenient and fast prototyping

---

A hyperparameter optimization and meta-learning toolbox for convenient and fast prototyping of machine-learning models.

Master status:
Dev status:
Code quality:
Latest versions:

For more information, visualization and details about the API check out the
website

---

Main features • Installation • Roadmap • License

---

Main features

• Thoroughly tested code base
• Compatible with any python machine-learning framework
• Optimize:
  • Anything from simple models
    to complex machine-learning-pipelines
  • Multi-level ensembles
  • Deep neural network architecture
  • Other optimization techniques (meta-optimization)
  • Or any function you can specify with this API
• Utilize state of the art optimization techniques like:
  • Simulated annealing
  • Evolution strategy
  • Bayesian optimization
• High performance: Optimizer time is neglectable for most models
• Choose from a variety of different optimization extensions to improve the optimization

Optimization Techniques	Tested and Supported Packages	Optimization Extentions
Local Search: Hill Climbing Stochastic Hill Climbing Tabu Search Random Methods: Random Search Random Restart Hill Climbing Random Annealing [*] Markov Chain Monte Carlo: Simulated Annealing Stochastic Tunneling Parallel Tempering Population Methods: Particle Swarm Optimizer Evolution Strategy Sequential Methods: Bayesian Optimization	Machine Learning: Scikit-learn XGBoost LightGBM CatBoost RGF Mlxtend Deep Learning: Tensorflow Keras Distribution: Multiprocessing Ray	Position Initialization: Scatter-Initialization [*] Warm-start Resource Allocation: Memory Short term Long term

Installation

The most recent version of Hyperactive is available on PyPi:

pip install hyperactive

Roadmap

v2.0.0:heavy_check_mark:

• Change API
• Ray integration

v2.1.0

• Save memory of evaluations for later runs (long term memory)
• Warm start sequence based optimizers with long term memory

v2.2.0

• Tree-structured Parzen Estimator
• Spiral optimization
• Downhill-Simplex-Method

v2.3.0

• Helper-classes for model pruning
• Helper-classes for dataset approximation

Experimental algorithms

The following algorithms are of my own design and, to my knowledge, do not yet exist in the technical literature. If any of these algorithms already exist I would like you to share it with me in an issue.

Random Annealing

A combination between simulated annealing and random search.

Scatter Initialization

Inspired by hyperband optimization.

References

[1] Proxy Datasets for Training Convolutional Neural Networks

License