HyperNOs Documentation

Introduction

HyperNOs is a Python project focused on the implementation of completely automatic, distributed and parallel neural operators hyperparameter optimization. The project aims to provide a framework for training neural operator models using Pytorch and Ray Tune for hyperparameter tuning. The library is designed to be highly flexible, making it easy to use with any kind of model and dataset. In the context of Neural Operators, where architecture design is still an active area of research, performing extensive hyperparameter optimization is crucial to obtain state-of-the-art results.

For a more detailed explanation of the library and its capabilities, please refer to our article: HyperNOs: Automated and Parallel Library for Neural Operators Research.

Supported Libraries

HyperNOs allows users to easily integrate and use models from popular neural-operator libraries or custom models. Is also very flexible and can be used with many different datasets.

I already implemented some examples of usage with the following popular libraries:

• NeuralOperator: Implement neural operator architectures like FNO, SFNO, TFNO, UNO, UQNO, GINO, FNOGNO, LocalNO, RNO, CODANO, OTNO.
• DeepXDE: Implement operator learning models like DeepONet, MIONet, POD-DeepONet, POD-MIONet.

You can find examples of how to use these models in the hypernos/examples directory with two dedicated subdirectories: deepxde_lib and neuralop_lib. There are implemented examples both for training a given architecture and for hyperparameter optimization routines.

Visualization website

The project also includes a visualization website: (https://hypernos.streamlit.app) that allows users to visualize the results obtained with HyperNOs library.

Installation

To set up the HyperNOs project, follow these steps:

1. Clone the repository:

   git clone --depth=1 https://github.com/MaxGhi8/HyperNOs.git
   cd HyperNOs
   

2. Install the required dependencies. It is recommended to create a virtual environment before installing the dependencies; I personally use pyenv (but others, like uv, are fine) and Python version 3.12.7 for this purpose:

   pyenv install 3.12.7
   pyenv virtualenv 3.12.7 hypernos
   pyenv activate hypernos
   

   Install in editable mode

   pip install -e .

   
   Installing the package makes it available as `hypernos` across your system and sets up the `hypernos-run` command.
   
   > [!WARNING]
   > For PyTorch, more attention may be needed during installation. We highly recommend following the [official documentation](https://pytorch.org/get-started/locally/) to install the correct version for your system (e.g., matching your CUDA version).
   
   

3. Download the dataset using the download_data.sh script:

   ./download_data.sh
   

4. Download pre-trained models (optional):

    git clone --depth=1 https://github.com/MaxGhi8/tests
   

   The previous repository contains the Tensorboard support for every model, the information about the training and the architecture's hyperparameters chosen. Then you can download running the following script and select the model that you want to download:

   ./download_trained_model.sh
   

Usage

After installation, you can run the provided examples in the neural_operators/examples directory.

Interactive Tutorials

We provide interactive Jupyter Notebooks in the notebook/ directory to help you get started:

• Training Tutorial: Learn how to train a Neural Operator.
• Ray Tune Tutorial: Learn how to tune hyperparameters of a Neural Operator.

Basic Training

To train a model (e.g., FNO) on a single machine, simply run the corresponding python script:

cd neural_operators/examples/
python train_fno.py

Python API

You can now import HyperNOs directly in your own scripts:

import torch
from hypernos.examples.train_fno import train_fno

# Run a training session
train_fno("poisson", "best", "L2")

Hyperparameter Optimization with Ray Tune

You can use Ray Tune to optimize hyperparameters.

Local Machine

To run Ray Tune on your local machine, first start a Ray head node:

ray start --head

Then run the Ray script:

cd neural_operators/examples/
python ray_fno.py

Cluster (Slurm)

For running on a cluster using Slurm, refer to SLURM_USAGE.md for instructions on using template.slurm.

Citation

If you use our library please consider citing our paper:

@misc{ghiotto2025hypernosautomatedparallellibrary,
      title={HyperNOs: Automated and Parallel Library for Neural Operators Research},
      author={Massimiliano Ghiotto},
      year={2025},
      eprint={2503.18087},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2503.18087},
}