zellij 0.0.4

A software framework for HyperParameters Optimization

Zellij is an open source Python framework for HyperParameter Optimization (HPO) which was orginally dedicated to Fractal Decomposition based algorithms [1] [2]. It includes tools to define mixed search space, manage objective functions, and a few algorithms. To implements metaheuristics and other optimization methods, Zellij uses DEAP[3] for the Evolutionary Algorithms part and BoTorch [4] for Bayesian Optimization. Zellij is defined as an easy to use and modular framework, based on Python object oriented paradigm.

Install Zellij

Original version

$ pip install zellij

Distributed Zellij

This version requires MPI, such as MPICH or Open MPI. It is based on mpi4py

$ pip install zellij[mpi]

User will then be able to use the MPI option of the Loss decorator.

@Loss(MPI=True)

Then the python script must be executed using mpiexec:

$ mpiexec -machinefile <path/to/hostfile> -n <number of processes> python3 <path/to/python/script>

Be carefull, before using this version, one must be familiar to MPI.

Quickstart

Define your search space

from zellij.core.search_space import Searchspace

labels = ["a","b"]
types = ["R","R"]
values = [[-5, 5],[-5, 5]]
neighborhood = [0.5,0.5]
sp = Searchspace(labels,types,values, neighborhood)

Define your loss function

import numpy as np
from zellij.core.loss_func import Loss

@Loss(save=False, verbose=True)
def himmelblau(x):
  x_ar = np.array(x)
  return np.sum(x_ar**4 -16*x_ar**2 + 5*x_ar) * (1/len(x_ar))

print(himmelblau)

Choose an optimization algorithm

from zellij.strategies.genetic_algorithm import Genetic_algorithm
ga = Genetic_algorithm(himmelblau, sp, 1000, pop_size=25, generation=40)
ga.run()
ga.show()

Dependencies

Original version

• Python >=3.6
• numpy=>1.21.4
• DEAP>=1.3.1
• botorch>=0.6.3.1
• gpytorch>=1.6.0
• matplotlib>=3.5.0
• seaborn>=0.11.2
• pandas>=1.3.4

MPI version

• Python >=3.6
• numpy=>1.21.4
• DEAP>=1.3.1
• botorch>=0.6.3.1
• gpytorch>=1.6.0
• matplotlib>=3.5.0
• seaborn>=0.11.2
• pandas>=1.3.4
• mpi4py>=3.1.2

Contributors

Design

• Thomas Firmin: [e-mail]
• El-Ghazali Talbi: [e-mail]

References

[1] Nakib, A., Ouchraa, S., Shvai, N., Souquet, L. & Talbi, E.-G. Deterministic metaheuristic based on fractal decomposition for large-scale optimization. Applied Soft Computing 61, 468–485 (2017).

[2] Demirhan, M., Özdamar, L., Helvacıoğlu, L. & Birbil, Ş. I. FRACTOP: A Geometric Partitioning Metaheuristic for Global Optimization. Journal of Global Optimization 14, 415–436 (1999).

[3] Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner, Marc Parizeau and Christian Gagné, "DEAP: Evolutionary Algorithms Made Easy", Journal of Machine Learning Research, vol. 13, pp. 2171-2175, jul 2012.

[4] M. Balandat, B. Karrer, D. R. Jiang, S. Daulton, B. Letham, A. G. Wilson, and E. Bakshy. BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. Advances in Neural Information Processing Systems 33, 2020.