pynxtools 0.2.1

Extend NeXus for experiments and characterization in Materials Science and Materials Engineering and serve as a NOMAD parser implementation for NeXus.

pynxtools is a tool designed for making your experimental data FAIR. It allows to develop ontologies and to create ontological instances based on the NeXus format.

Installation

It is recommended to use python 3.10 with a dedicated virtual environment for this package. Learn how to manage python versions and virtual environments.

Install the latest stable version of this package from PyPI with

pip install pynxtools

You can also install the latest development version with

pip install git+https://github.com/FAIRmat-NFDI/pynxtools.git

Scope

pynxtools (previously called nexusutils) is intended as a parser for combining various instrument output formats and electronic lab notebook (ELN) formats to an hdf5 file according to NeXus application definitions.

Additionally, the software is used in the research data management system NOMAD for making experimental data searchable and publishable. NOMAD is developed by the FAIRMAT consortium, as a part of the German National Research Data Infrastructure (NFDI).

The software tools are located inside pynxtools and they are shipped with unit tests located in tests. Some examples with real datasets are provided in examples. It guides you through the process of converting instrument raw data into the NeXus standard and visualising the files content.

Command line tools

• dataconverter: Creates compliant instances of NeXus/HDF5 files to NeXus schemas.
• read_nexus: Outputs a debug log for a given NeXus file.
• generate_eln: Outputs ELN files that can be used to add metadata to the dataconverter routine.

Documentation

Documentation for the different tools can be found here.

Plugins

There are a number of plugins available for pynxtools. These are extensions of pynxtools used for reading data of specialized experimental techniques.

• pynxtools-mpes: A reader for multi-dimensional photoelectron spectroscopy data.
• pynxtools-stm: A reader for scanning tunneling microscopy (SPM) and spectroscopy (STS) data.
• pynxtools-xps: A reader for X-ray photoelectron spectroscopy (XPS) data.

Respective readers for the research fields of electron microscopy and atom probe are currently refactored into pynxtools plugins. Until this refactoring will have become completed, users are advised to use the apm and em readers via pynxtools<=0.1.1.

• pynxtools-apm: A reader for atom probe as well as related field ion microscopy data.
• pynxtools-em: A reader for electron microscopy data.

You can install each of the plugins together with pynxtools by passing the name of the plugin as an extra to the pip install call. For example, for the pynxtools-mpes plugin:

pip install pynxtools[mpes]

In addition, you can also install all of the pynxtools reader plugins which are maintained by FAIRmat by passing the [convert] extra to the pip install call:

pip install pynxtools[convert]

There is also a cookiecutter template available for creating your own pynxtools plugin.

Contributing

Development install

Install the package with its dependencies:

git clone https://github.com/FAIRmat-NFDI/pynxtools.git \\
    --branch master \\
    --recursive pynxtools
cd pynxtools
git submodule sync --recursive
git submodule update --init --recursive --jobs=4
python -m pip install --upgrade pip
python -m pip install -e .
python -m pip install -e ".[dev]"

There is also a pre-commit hook available which formats the code and checks the linting before actually commiting. It can be installed with

pre-commit install

from the root of this repository.

Test this software

Especially relevant for developers, there exists a basic test framework written in pytest which can be used as follows:

python -m pytest -sv tests

Run examples

A number of examples exist which document how the tools can be used. For a standalone usage convenient jupyter notebooks are available for each tool. To use them jupyter and related tools have to be installed in the development environment as follows:

python -m pip install jupyter
python -m pip install jupyterlab
python -m pip install jupyterlab_h5web

Questions, suggestions?

To ask further questions, to make suggestions how we can improve these tools, to get advice on how to build on this work, or to get your parser included into NOMAD, you can:

• Open an issue on the pynxtools GitHub
• Use our forums at matsci.org
• Write to [e-mail]
• Contact directly the lead developers of the individual parsers.

Does this software require NOMAD or NOMAD OASIS ?

No. The data files produced here can be uploaded to Nomad. Therefore, this acts like the framework to design schemas and instances of data within the NeXus universe.

Troubleshooting

Please check this guide for any issues you face with the tool. If you don't find a solution there, please make a new Github Issue.