nomad-simulations

The nomad-simulations is an open-source Python package for managing Materials Science simulation data. It is following the plugin architechture of NOMAD. This package contains a set of section definitions (Python classes) with quantities (attributes) and methods defined to automate data extraction from different simulation codes. These section definitions can be used at any prefered level by the user, they can be modified and extended, and we welcome external collaborators.

Read more in the official documentation page.

Getting started

nomad-simulations can be installed as a PyPI package using pip:

[!WARNING] Unfortunately, the current plugin mechanism is not supported by the latest nomad-lab version on PyPI and therefore an index url pointing to the NOMAD Gitlab registry needs to be added.

pip install nomad-simulations --index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple

Development

If you want to develop locally this package, clone the project and in the workspace folder, create a virtual environment (you can use Python 3.9, 3.10, or 3.11):

git clone https://github.com/nomad-coe/nomad-simulations.git
cd nomad-simulations
python3.11 -m venv .pyenv
. .pyenv/bin/activate

Make sure to have pip upgraded:

pip install --upgrade pip

We recommend installing uv for fast pip installation of the packages:

pip install uv

Install the nomad-lab package:

uv pip install '.[dev]'

The plugin is still under development. If you would like to contribute, install the package in editable mode (with the added -e flag):

uv pip install -e '.[dev]'

Note: If you do not use uv, you need to add the --index-url flag as in the PyPI command for installing the package:

pip install -e '.[dev]' --index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple

Run the tests

You can run locally the tests:

python -m pytest -sv tests

where the -s and -v options toggle the output verbosity.

Our CI/CD pipeline produces a more comprehensive test report using the pytest-cov package. You can generate a local coverage report:

uv pip install pytest-cov
python -m pytest --cov=src tests

You can also run the script to generate a local file coverage.txt with the same information:

./scripts/generate_coverage_txt.sh

Run linting and auto-formatting

We use Ruff for linting and formatting the code. Ruff auto-formatting is also a part of the GitHub workflow actions. You can run locally:

ruff check .
ruff format . --check

Debugging

For interactive debugging of the tests, use pytest with the --pdb flag. We recommend using an IDE for debugging, e.g., VSCode. If that is the case, add the following snippet to your .vscode/launch.json:

{
  "configurations": [
      {
        "name": "<descriptive tag>",
        "type": "debugpy",
        "request": "launch",
        "cwd": "${workspaceFolder}",
        "program": "${workspaceFolder}/.pyenv/bin/pytest",
        "justMyCode": true,
        "env": {
            "_PYTEST_RAISE": "1"
        },
        "args": [
            "-sv",
            "--pdb",
            "<path-to-plugin-tests>",
        ]
    }
  ]
}

where <path-to-plugin-tests> must be changed to the local path to the test module to be debugged.

The settings configuration file .vscode/settings.json automatically applies the linting and formatting upon saving the modified file.

Documentation on Github pages

To view the documentation locally, install the related packages using:

uv pip install -r requirements_docs.txt

Run the documentation server:

mkdocs serve

Adding this plugin to NOMAD

Currently, NOMAD has two distinct flavors that are relevant depending on your role as an user:

1. A NOMAD Oasis: any user with a NOMAD Oasis instance.
2. Local NOMAD installation and the source code of NOMAD: internal developers.

Adding this plugin in your NOMAD Oasis

Read the NOMAD plugin documentation for all details on how to deploy the plugin on your NOMAD instance.

Adding this plugin in your local NOMAD installation and the source code of NOMAD

Modify the script under /nomad/scripts/install_default_plugins.sh and add the path to this repository pointing to the @develop branch:

<other-content-in-install_default_plugins.sh...>
git+https://github.com/nomad-coe/nomad-simulations.git@develop

Then, go to your NOMAD folder, activate your NOMAD virtual environment and run:

deactivate
cd <route-to-NOMAD-folder>/nomad
source .pyenv/bin/activate
./scripts/setup_dev_env.sh

Alternatively and only valid for your local NOMAD installation, you can modify nomad.yaml to include this plugin:

plugins:
  entry_points:
    include:
      - ["nomad_simulations.schema_packages:nomad_simulations_plugin"]

Note! Once you modify your nomad.yaml file adding include, all the default plugins will be disconnected, so you will need to include them as well.

How to cite this work

Pizarro, J.M., Boydas, E.B., Daelman, N., Ladines, A.N., Mohr, B. & Rudzinski, J.F., NOMAD Simulations [Computer software]. https://zenodo.org/doi/10.5281/zenodo.13838811

Main contributors