hyperactive 2.0.0b0

A hyperparameter optimization toolbox for convenient and fast prototyping

---

A hyperparameter optimization and meta-learning toolbox for convenient and fast prototyping of machine-learning models.

Master status:
Dev status:
Code quality:
Latest versions:

---

Documentation • Main features • Installation • Roadmap • License

---

Main features

• Thoroughly tested code base
• Compatible with any python machine-learning framework
• Optimize:
  • Anything from simple models
    to complex machine-learning-pipelines
  • Multi-level ensembles
  • Deep neural network architecture
  • Other optimization techniques (meta-optimization)
  • Or any function you can specify with this API
• Utilize state of the art optimization techniques like:
  • Simulated annealing
  • Evolution strategy
  • Bayesian optimization
• High performance: Optimizer time is neglectable for most models
• Choose from a variety of different optimization extensions to improve the optimization

Optimization Techniques	Tested and Supported Packages	Optimization Extentions
Local Search: Hill Climbing Stochastic Hill Climbing Tabu Search Random Methods: Random Search Random Restart Hill Climbing Random Annealing [*] Markov Chain Monte Carlo: Simulated Annealing Stochastic Tunneling Parallel Tempering Population Methods: Particle Swarm Optimizer Evolution Strategy Sequential Methods: Bayesian Optimization	Machine Learning: Scikit-learn XGBoost LightGBM CatBoost RGF Mlxtend Deep Learning: Tensorflow Keras Distribution: Multiprocessing Ray	Position Initialization: Scatter-Initialization [*] Warm-start Resource Allocation: Memory Short term Long term

This readme provides only a short introduction. For more information check out the
full documentation

Installation

The most recent version of Hyperactive is available on PyPi:

pip install hyperactive

Roadmap

v2.0.0

• Change API
• Ray integration

v2.1.0

• Save memory of evaluations for later runs (long term memory)
• Warm start sequence based optimizers with long term memory

v2.2.0

• Tree-structured Parzen Estimator
• Spiral optimization
• Downhill-Simplex-Method

Experimental algorithms

The following algorithms are of my own design and, to my knowledge, do not yet exist in the technical literature. If any of these algorithms still exist I ask you to share it with me in an issue.

Random Annealing

A combination between simulated annealing and random search.

Scatter Initialization

Inspired by hyperband optimization.

References

[1] Proxy Datasets for Training Convolutional Neural Networks

License