hyperactive 1.0.0

A hyperparameter optimization toolbox for convenient and fast prototyping

---

A hyperparameter optimization and meta-learning toolbox for convenient and fast prototyping of machine-learning models.

Master status:
Dev status:
Code quality:

---

Documentation | Main features | Installation | License

---

Main features

• Very simple but versatile API
• Thoroughly tested code base
• Compatible with any python machine-learning framework
• Optimize:
  • Anything from simple models
    to complex machine-learning-pipelines
  • Multi-level ensembles
  • Deep neural network architecture
  • Other optimization techniques (meta-optimization)
  • Or any function you can specify with this API
• Utilize state of the art optimization techniques like:
  • Simulated annealing
  • Evolution strategy
  • Bayesian optimization
• High performance: Optimizer time is neglectable for most models
• Choose from a variety of different optimization extensions to improve the optimization

Optimization Techniques	Tested and Supported Packages	Optimization Extentions
Local Search: Hill Climbing Stochastic Hill Climbing Tabu Search Random Methods: Random Search Random Restart Hill Climbing Random Annealing [*] Markov Chain Monte Carlo: Simulated Annealing Stochastic Tunneling Parallel Tempering Population Methods: Particle Swarm Optimizer Evolution Strategy Sequential Methods: Bayesian Optimization	Machine Learning: Scikit-learn XGBoost LightGBM CatBoost Deep Learning: Keras Distribution: Multiprocessing	Position Initialization: Scatter-Initialization [*] Warm-start Meta-Learn (coming soon) Resource Allocation: Memory Proxy Datasets [1] (coming soon)

This readme provides only a short introduction. For more information check out the
full documentation

Installation

The most recent version of Hyperactive is available on PyPi:

pip install hyperactive

Experimental algorithms

The following algorithms are of my own design and, to my knowledge, do not yet exist in the technical literature. If any of these algorithms still exist I ask you to share it with me in an issue.

Random Annealing

A combination between simulated annealing and random search.

Scatter Initialization

Inspired by hyperband optimization.

References

[1] Proxy Datasets for Training Convolutional Neural Networks

[2] An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks

License