marisco 0.9.5

MARIS companion package and tutorials

MARISCO

The IAEA Marine Radioactivity Information System (MARIS) provides open access to radioactivity measurements in marine environments. Developed by the IAEA Marine Environmental Laboratories in Monaco, MARIS offers data on seawater, biota, sediment, and suspended matter.

This Python package includes command-line tools to convert MARIS datasets into NetCDF or .csv formats, enhancing compatibility with various scientific and data analysis software.

Core Concept: Handlers

marisco is built around the concept of handlers - specialized modules designed to convert MARIS datasets into NetCDF format. Each handler is tailored to a specific data provider and implemented as a dedicated Jupyter notebook.

Literate Programming Approach

We’ve adopted a Literate Programming approach, which means:

1. Documentation: Each handler serves as comprehensive documentation.
2. Code Reference: The notebooks contain the actual implementation code.
3. Communication Tool: They facilitate discussions with data providers about discrepancies or inconsistencies.

Powered by nbdev

To achieve this, we leverage nbdev, a powerful tool that allows us to:

1. Write code within Jupyter notebooks
2. Automatically export relevant parts as dedicated Python modules

This approach bridges the gap between documentation and implementation, ensuring they remain in sync.

See It in Action

For a concrete example of this approach, check out our OSPAR dataset handler implementation.

List of currently available handlers

MARISCO includes a suite of specialized data handlers designed to:

• Convert provider-specific data formats into standardized MARIS NetCDF files
• Ensure data quality and consistency across providers
• Facilitate integration with the MARIS marine radioactivity database
• Support automated data processing workflows

The following handlers are currently implemented:

Handler	Description	Link to Data Source
MARIS Legacy	All legacy MARIS datasets from the MARIS Master Database	-
HELCOM	HELCOM marine environment protection datasets	HELCOM
OSPAR	OSPAR marine environment datasets	ODIMS OSPAR
TEPCO	TEPCO Fukushima monitoring data	TEPCO Monitoring
GEOTRACES	BODC GEOTRACES oceanographic data	GEOTRACES IDP2021

Install

Now, to install marisco simply run

pip install marisco

Once successfully installed, run the following command:

maris_init

This command:

1. creates a .marisco/ directory containing various configuration/configurable files ((below)) in your /home directory;
2. creates a configs.toml file containing default but configurable settings (default paths, …);
3. downloads several MARIS DB nomenclature/lookup table into .marisco/lut/ directory;
4. downloads maris-template.nc, the MARIS NetCDF4 template.

Zotero API key

Upon conversion, marisco will automatically retrieve the bibliographic metadata of each MARIS dataset from Zotero. To do so, you need to define the following environment variable ZOTERO_API_KEY containing the MARIS Zotero API key. Please contact the MARIS team to get your API key.

Getting started

Command line utilities

All commands accept a -h argument to get access to its documentation.

maris_init

Download configuration file, NetCDF MARIS template and required lookup tables (nomenclatures).

maris_to_nc

Convert helcom, geotraces, tepco or ospar marine radioactivity datasets to MARIS NetCDF4 format.

usage: maris_to_nc [-h] [--src SRC] ds dest

positional arguments:
  ds          Name of the dataset to encode as NetCDF4
  dest        Output path for NetCDF file

options:
  -h, --help  show this help message and exit
  --src SRC   Optional input data path only required for the 'GEOTRACES' dataset

For instance: maris_to_nc ospar 191-OSPAR-2024.nc

maris_db_to_nc

The MARIS Master Database integrates two types of datasets:

• Historical datasets retrieved from published scientific papers
• Ongoing monitoring data from international programs like HELCOM, OSPAR, TEPCO, and GEOTRACES

This command-line utility converts MARIS datasets from their legacy format to NetCDF4, making them more accessible for modern data analysis workflows. Users can either convert the entire database or specify particular datasets by their reference IDs for selective conversion.

usage: maris_db_to_nc [-h] [--ref_ids REF_IDS] src dest

Convert MARIS legacy database to NetCDF4 format. If ref_ids is provided as comma-separated values, only encodes those subsets.

positional arguments:
  src                Path to MARIS database dump as `.txt` file
  dest               Output path for NetCDF file(s)

options:
  -h, --help         show this help message and exit
  --ref_ids REF_IDS  Optional comma-separated reference IDs (e.g., "123,456,789") (default: )

For instance:

• maris_db_to_nc "~/pro/data/maris/2024-11-20 MARIS_QA_shapetype_id=1.txt" ~/pro/tmp/output
• or maris_db_to_nc "~/pro/data/maris/2024-11-20 MARIS_QA_shapetype_id=1.txt" ~/pro/tmp/output --ref_ids="16,30" for a subset of the MARIS Master Database.

Development

The MARIS NetCDF template is generated from nbs/files/cdl/maris.cdl Common Data Language (CDL) file as defined by Unidata. To generate the MARIS NetCDF template nbs/files/nc/maris-template.nc, install the NetCDF-C utilities, once in Marisco home directory, run:

ncgen -4 -o nc/maris-template.nc cdl/maris.cdl