Convolutional Neural Networks for Semi-discrete NSE

A python module with

• a dynamic setup of Convolutional Neural Networks in PyTorch
• an interface to FEniCS to generate data from FEM simulations of flows and
• a numerical realization of FEM norm in the training neural networks

developed to design very low-dimensional LPV approximations of incompressible Navier-Stokes equations.

The Module

Installation

# package for sparse cholesky factorizations 
# not needed but speed up with FEM norms and POD

apt install libsuitesparse-dev
pip install scikit-sparse==0.4.5

# fenics -- for the FEM part
apt install fenics  # see https://fenicsproject.org/download/

# install this module and helper modules
pip install .  # if to be installed from source
# pip install nse_nn_lpv  # from the pypi index

Documentation

The underlying principles are explained in the paper

Benner, Heiland, Bahmani (2022): Convolutional Neural Networks for Very Low-dimensional LPV Approximations of Incompressible Navier-Stokes Equations

that appeared in Frontiers.

This module was the backend of the presented numerical examples.

Reproducing numerical results

Generally, the numerical procedure consists of

1. generating FEM simulation data
2. setting up and training the neural networks
3. running simulations

Step 1. is defined in the folder simulations-training-data. See the corresponding (simulation) RUNME.md

Steps 2. and 3. in the folder tests. See the corresponding (training) RUNME sh for instructions.

Note: The folders and the numerical scripts are part of the complete sources in the git repository under the tag that corresponds to the paper under question or in the dedicated Zenodo repository. The pypi distribution only contains the core module.

Data and Handling

Generally, the data is

1. generated by the simulation
2. interpolated to the pictures
3. imported to PyTorch as tensors

The relevant formats and routines are as follows:

import nse_nn_lpv.nse_data_helpers as ndh

1. time_dep_nse_generic.py computes the trajectories with snapshots vvec and interpolates them to the two pictures vmatx and vmaty at all time instances ti. The data is stored as a json file, say, data.json like

{ti: {'vvec': vvec, 'vmatx': vmatx, 'vmaty': vmaty},
 'femdata': 'information of the simulation ...'}

2. (datal, vvecl) = ndh.get_nse_img_data('data.json') takes the data file data.json and returns the data as a list (of tuples) of numpy arrays:

datal = [(vmatx, vmaty, t_0), ..., (vmatx, vmaty, t_end)]
vvecl = [vvec_0, ..., ..., ..., ..., vvec_end]

3. trn_nse_data = ndh.NSEDataset(nsedatal, vvecl) takes the data lists and makes it available as a pytorch data set. In particular, the two pictures vmatx, vmaty are merged into a tensor.

4. stst_dataloader = pytorch.DataLoader(trn_nse_data, batch_size=1, shuffle=True) then defines a way to access the data. E.g.,

(ttstset, tstvec) = next(iter(stst_dataloader))

returns a data point via

• ttstset: the tensor of size (batch_size, 2, width, height), where (width, height) are the dimensions of the pictures
• tstvec: the corresponding vvecs (needed, e.g., for the loss function later)

Python Machine-Learning Resources

• an overview
  1. Tensorflow -- see below
  2. Pytorch -- see below
  3. NeuroLab
  4. ffnet
  5. Scikit-Neural Network
  6. Lasagne
  7. pyrenn

Tensorflow

• website
• Article for understanding NN using TensorFlow
• Previous experience with building NN
• Visualization feature TensorBoard
• Based on keras

PyTorch

• website
• Tutorial for simple NN
• Recommended by colleagues (Lessig, Richter)

Scikit-Learn

• website
• looks well maintained
• many routines for data processing
• a few on neural network