Checkpointing functionality for DOLFINx meshes/functions with ADIOS2
Project description
ADIOS4DOLFINx - A framework for checkpointing in DOLFINx
ADIOS4DOLFINx is an extension for DOLFINx to checkpoint meshes, meshtags and functions using ADIOS 2.
The code uses the ADIOS2 Python-wrappers to write DOLFINx objects to file, supporting N-to-M (recoverable) and N-to-N (snapshot) checkpointing. See: Checkpointing in DOLFINx - FEniCS 23 or the examples in the Documentation for more information.
For scalability, the code uses MPI Neighbourhood collectives for communication across processes.
Statement of Need
As the usage of high performance computing clusters increases, more and more large-scale, long-running simulations are deployed.
The need for storing intermediate solutions from such simulations are crucial, as the HPC system might crash, or the simulation might crash or exceed the alloted computational budget.
Having a checkpoint of related variables, such as the solutions to partial differential equations (PDEs) is therefore essential.
The adios4dolfinx
library extends the DOLFINx computational framework for solving PDEs with checkpointing functionality, such that immediate solutions and mesh information can be stored and re-used in another simulation.
Installation
Compatibility with DOLFINx:
- ADIOS4DOLFINx v0.8.1 is compatible with DOLFINx v0.8.x
- ADIOS4DOLFINx v0.7.3 is compatible with DOLFINx v0.7.x
Dependencies
The library depends on the Python-interface of DOLFINx and an MPI-build of ADIOS2.
Therefore ADIOS2
should not be install through PYPI/pip, but has to be installed through Conda, Spack or from source.
Docker
An MPI build of ADIOS2 is installed in the official DOLFINx containers, and thus there are no additional dependencies required to install adios4dolfinx
on top of DOLFINx in these images.
Create a Docker container, named for instance dolfinx-checkpoint
.
Use the nightly
tag to get the main branch of DOLFINx, or stable
to get the latest stable release
docker run -ti -v $(pwd):/root/shared -w /root/shared --name=dolfinx-checkpoint ghcr.io/fenics/dolfinx/dolfinx:nightly
For the latest version compatible with nightly (with the ability to run the test suite), use
python3 -m pip install adios4dolfinx[test]@git+https://github.com/jorgensd/adios4dolfinx@main
If you are using the stable
image, you can install adios4dolfinx
from PYPI with
python3 -m pip install adios4dolfinx[test]
This docker container can be opened with
docker container start -i dolfinx-checkpoint
at a later instance
Conda
[!NOTE]
Conda supports the stable release of DOLFINx, and thus the appropriate version should be installed, see the section above for more details.
Following is a minimal recipe of how to install adios4dolfinx, given that you have conda installed on your system.
conda create -n dolfinx-checkpoint python=3.10
conda activate dolfinx-checkpoint
conda install -c conda-forge adios4dolfinx
[!NOTE] Remember to download the appropriate version of
adios4dolfinx
from Github adios4dolfinx: Releases
To run the test suite, you should also install ipyparallel
, pytest
and coverage
, which can all be installed with conda
conda install -c conda-forge ipyparallel pytest coverage
Functionality
DOLFINx
- Reading and writing meshes, using
adios4dolfinx.read/write_mesh
- Reading and writing meshtags associated to meshes
adios4dolfinx.read/write_meshtags
- Reading checkpoints for any element (serial and parallel, arbitrary number of functions and timesteps per file). Use
adios4dolfinx.read/write_function
. - Writing standalone function checkpoints relating to "original meshes", i.e. meshes read from
XDMFFile
. Useadios4dolfinx.write_function_on_input_mesh
for this. - Store mesh partitioning and re-read the mesh with this information, avoiding calling SCOTCH, Kahip or Parmetis.
[!IMPORTANT]
For checkpoints written withwrite_function
to be valid, you first have to store the mesh withwrite_mesh
to the checkpoint file.
[!IMPORTANT]
A checkpoint file supports multiple functions and multiple time steps, as long as the functions are associated with the same mesh
[!IMPORTANT]
Only one mesh per file is allowed
Example Usage
The repository contains many documented examples of usage, in the docs
-folder, including
- Reading and writing mesh checkpoints
- Storing mesh partitioning data
- Writing mesh-tags to a checkpoint
- Reading and writing function checkpoints
- Checkpoint on input mesh Further examples can be found at ADIOS4DOLFINx examples
Backwards compatibility
[!WARNING] If you are using v0.7.2, you are adviced to upgrade to v0.7.3, as it contains som crucial fixes for openmpi.
Legacy DOLFIN
Only checkpoints for Lagrange
or DG
functions are supported from legacy DOLFIN
- Reading meshes from the DOLFIN HDF5File-format
- Reading checkpoints from the DOLFIN HDF5File-format (one checkpoint per file only)
- Reading checkpoints from the DOLFIN XDMFFile-format (one checkpoint per file only, and only uses the
.h5
file)
See the API for more information.
Testing
This library uses pytest
for testing.
To execute the tests, one should first install the library and its dependencies, as listed above.
Then, can execute all tests by calling
python3 -m pytest .
Testing against data from legacy dolfin
Some tests check the capability of reading data created with the legacy version of DOLFIN. To create this dataset, start a docker container with legacy DOLFIN, for instance:
docker run -ti -v $(pwd):/root/shared -w /root/s
hared --rm ghcr.io/scientificcomputing/fenics:2024-02-19
Then, inside this container, call
python3 ./tests/create_legacy_data.py --output-dir=legacy
Testing against data from older versions of ADIOS4DOLFINx
Some tests check the capability to read data generated by adios4dolfinx<0.7.2
.
To generate data for these tests use the following commands:
docker run -ti -v $(pwd):/root/shared -w /root/shared --rm ghcr.io/fenics/dolfinx/dolfinx:v0.7.3
Then, inside the container, call
python3 -m pip install adios4dolfinx==0.7.1
python3 ./tests/create_legacy_checkpoint.py --output-dir=legacy_checkpoint
Long term plan
The long term plan is to get this library merged into DOLFINx (rewritten in C++ with appropriate Python-bindings).
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for adios4dolfinx-0.8.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7631ee017412e6e57b7efa954d9c637e79c40497e9946565cebb6896282a6939 |
|
MD5 | ff64181c9724e9b963e06c7a5c028748 |
|
BLAKE2b-256 | 3bed346d52fcf5f6ffc33c754f5a1c38a8c56654277a09662c522c10b46d4118 |