dragon-autodl 1.0

Implementation of the algorithmic framework from <An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters>

DRAGON, for DiRected Acyclic Graphs OptimizatioN, is an open source Python package for the optimization of Deep Neural Networks Hyperparameters and Architecture [1]. DRAGON is not a no code package, but you can get familiar with it quickly thanks to the Documentation.

Key Features

• A flexible seach space

  • The search space based on Directed Acyclic Graphs (DAGs) where the nodes can be any PyTorch layer (custom or not) and the edges are the connections between them.
  • The code to implement the DAGs-based search space was inspired by the zellij package developed for hyperparameters optimization.
  • DRAGON search space includes cell-based search spaces [4]_.

• Flexible optimization algorithms

  • The search algorithms defined in DRAGON are based on search operators used to modify elements of the search space (e.g., mutations, neighborhoods, crossover), which can be used to develop new search algorithms.
  • Efficient algorithms are also implemented in DRAGON such as the Random Search, Evolutionary Algorithm, Mutant UCB, and HyperBand.

• Applications to various tasks

  • The flexibility of DRAGON makes it usable for various applications.
  • For example: image classification, time series forecasting, electricity consumption forecasting, wind power forecasting or tabular data.

• Easy parallelization over multiple GPUs

  • The distributed version requires a MPI library, such as MPICH or Open MPI and is based on mpi4py.

Basic Concepts

• The Search Space is a mix-variable search space. Numerical, categorical and graph objects may be jointly optimized. Each object is associated with a variable, which defines what values an object can take.
• Base on this search space, several Search Operators are defined, showing how the objects can be manipulate to find the neighboring values.

Install DRAGON

Basic version

After cloning the git repository, install DRAGON, using:

 pip install -e dragon

Distributed version

If you plan on using the distributed version, you have to install the mpi4py package:

 pip install mpi4py

Documentation

Additional dependencies are required to run the documentation notebooks:

 pip install -e dragon[docs]

Dependencies

• Python >=3.9
• numpy<2.0.0
• torch
• graphviz
• pandas
• [mpi]:
  • mpi4py
• [docs]:
  • openml
  • sklearn
  • optuna
  • matplotlib
  • skorch

Contributors

Design

• Julie Keisler: [e-mail]

References

[1] Keisler, J., Talbi, E. G., Claudel, S., & Cabriel, G. (2024). An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters. Journal of Machine Learning Research.