nomad-fsspec

An fsspec filesystem for the raw data of NOMAD, the materials science repository. It is read-only.

import fsspec

fs = fsspec.filesystem("nomad")  # https://nomad-lab.eu/prod/v1 by default
fs.ls("/")  # two folders: /datasets and /uploads
fs.ls("/datasets")  # public datasets
fs.ls("/datasets/wWgAnNZNQxOHLf97H62dgw", detail=False)  # the uploads of one dataset, by id
fs.upload_names(["Xa2LB__ESiKu-odRBPJhjA"])  # {"Xa2LB__ESiKu-odRBPJhjA": "nomad-demo-data.zip"}
fs.get_file(
    "/datasets/wWgAnNZNQxOHLf97H62dgw/Xa2LB__ESiKu-odRBPJhjA/nomad-demo-data/BrK_svSe/TBCC006.ABC/vasprun.xml.relax2",
    "vasprun.xml",
)

The top level has two folders, so a path means the same thing with or without a token:

• /datasets/<dataset id>/<upload id>/<path inside the upload> for public datasets. Datasets have their names in display_name. Uploads inside a dataset are listed by id, since naming all uploads of a large dataset at once makes NOMAD time out; upload_names looks names up 25 at a time and keeps them, and search_uploads finds the ids of uploads by name.
• /uploads/<upload id>/<path inside the upload> for your uploads and those shared with you, published or not, as NOMAD's uploads page lists them. Listing /uploads needs a token. Published uploads can be read through it without one.

fs = fsspec.filesystem("nomad", token="...")
fs.ls("/uploads")  # your uploads, with their names in display_name

A personal access token with the uploads:read scope is enough. To use a NOMAD Oasis, pass base_url:

fs = fsspec.filesystem("nomad", base_url="https://my-oasis.example.org/nomad-oasis")

NOMAD's rate limits (HTTP 429 and 503) are retried. Redirects are not followed: a base URL that redirects is reported with the address it redirects to.

Development

uv sync
uv run pytest -m "not live"  # unit tests
uv run pytest -m live        # a few calls to nomad-lab.eu
uv run ruff check . && uv run ruff format --check . && uv run mypy src

License

MIT