MARISCO

The IAEA Marine Radioactivity Information System (MARIS) provides open access to radioactivity measurements in marine environments (seawater, biota, sediment, and suspended matter) collected by national laboratories, monitoring programmes, and research institutions worldwide. Developed by the IAEA Marine Environmental Laboratories in Monaco, MARIS is the authoritative international reference for marine radioactivity data.

marisco is being developed as the data processing foundation of MARIS: a Python package that converts provider datasets into standardised NetCDF4 and CSV formats for ingestion into the MARIS database.

How marisco works

MARIS does not impose a data schema on providers. It accepts data in whatever form it arrives, and it makes the cost of that acceptance visible.

Each dataset is processed by a dedicated handler, a Jupyter notebook that documents every curation decision alongside the code that implements it. Unit conversions, nuclide nomenclature mapping, coordinate standardisation, outlier flags: every choice is visible, traceable, and auditable. Some formats need less work than others, and the handler makes that difference plain. If you are a data provider, see contributing data for what this means for your dataset.

This is deliberate. The friction a dataset creates is a signal, and surfacing it is the point. It tells the MARIS team how much work a provider’s format really requires, it gives the provider a concrete, itemised record of what to improve, and it makes the next release of the dataset cheaper to ingest. Over time, the easiest path for everyone is for providers to align with the MARIS way of organising data, not because it is imposed, but because the cost of not doing so is visible.

The payoff is a harmonised world reference: measurements of marine radioactivity from providers worldwide, stored in the MARIS database in one comparable form. That comparability is the real value, for data providers, for the scientific community, and beyond. It rests on visible curation: the process that made the data comparable is documented and auditable, so the harmonisation can be checked, questioned, and trusted.

Handlers are shared with data providers as a working document. If a conversion looks wrong, a mapping is missing, or terminology differs from your internal standards, the notebook is where we discuss and resolve it, not buried in email threads or internal comments.

For a concrete example, see the GEOTRACES handler.

The harmonised data reaches users through three channels: the explore portal for interactive visualisation, the datasets page for downloading individual NetCDF files (joined soon by INIS), and the MARIS API for programmatic access.

Available handlers

Install

pip install marisco

Two environment variables may be required depending on the dataset:

• ZOTERO_API_KEY: used to fetch bibliographic metadata from the MARIS Zotero library. Contact MARIS Administrators to obtain your key:

  export ZOTERO_API_KEY=your_api_key_here
  

• INIS: bibliographic metadata is being progressively migrated from Zotero to the IAEA INIS database. The INISClient fetches records via curl, which must be installed separately. In the near term both systems coexist; INIS will supersede Zotero as the migration completes.

Quick start

Command line utilities

All commands accept -h for full documentation.

Convert a provider dataset to MARIS NetCDF4:

marisco-ingest helcom output/100-HELCOM-MORS-2024.nc

The handler is selected by name, and each handler decides whether it fetches data online or reads a local source. Run marisco-ingest with an invalid handler name to list the available handlers.

Convert the MARIS legacy database dump to NetCDF4:

marisco-ingest maris_legacy ~/output --src ~/data/maris/dump.txt

To encode only a subset of references:

marisco-ingest maris_legacy ~/output --src ~/data/maris/dump.txt --ref_ids "16,30"

Handlers that produce several files write them into the destination folder, which is created if missing.

Convert a NetCDF4 file to MARIS-format CSV (for database import):

marisco-export ~/output/191-OSPAR-2024.nc

The CSV utility appends the sample type automatically (e.g. 191-OSPAR-2024_BIOTA.csv), and may generate multiple files, one per sample type present in the source dataset. Use --dest to choose an output stem other than the source file name.

Documentation

Documentation is organised into two groups.

How-to guides

Practical walkthroughs for common tasks:

• Contributing data to MARIS: how a data provider delivers a dataset to the repository
• Writing a new handler: step-by-step guide to adding a new data provider to the marisco pipeline
• Nomenclature reconciliation: repeatable procedure for mapping provider names (nuclides, species, units, …) to MARIS standard identifiers
• Explore a MARIS NetCDF file: step-by-step guide to reading a MARIS NetCDF file: global metadata, group structure, lookup tables and nomenclatures, and each sample-type dataset as a pandas DataFrame

Reference

Detailed specifications and reference material:

• Reporting template: the MARIS data reporting template as documentation, with the general instructions, per-sample-type column tables, and lookup lists
• Field Definitions: complete field-by-field reference with MARISCO column names, NetCDF variable names, CSV variable names, types, and lookup tables
• MARIS database schema: the master database sample and measure tables, their mandatory columns, and their constraints

Development

marisco follows the nbdev workflow: all code lives in notebooks under nbs/, and the marisco/ package is generated from them.

• Edit the notebooks in nbs/, then run nbdev-export to regenerate the modules and nbdev-readme to regenerate this README.
• After changing pyproject.toml (entry points, dependencies), reinstall the package with pip install -e ..
• The MARIS NetCDF template is generated from nbs/files/cdl/maris.cdl. To regenerate it locally, install NetCDF-C and run:

ncgen -4 -o nbs/files/nc/maris-template.nc nbs/files/cdl/maris.cdl

• Contributor guidance on architecture, coding conventions, and handler documentation style lives in the CRAFTs/ notebooks at the repository root.
• Writing a new handler walks through adding a new data provider to the pipeline.

Acknowledgements

Development of this package was supported by the Solveit platform, an interactive development environment for dialog-driven software engineering.