ckanext-marinerg

CKAN extension powering the MARINERG-i data catalogue — the data catalogue for the MARINERG-i e-infrastructure, a Distributed Research Infrastructure for Marine Renewable Energy (MRE) test facilities across the EU. On top of stock CKAN it adds a marine metadata schema, controlled-vocabulary validation, Zenodo/DOI prefill, a facility/equipment picker, DCAT-AP 3.0 + FAIR Signposting export, Keycloak OIDC login, and MARINERG-i theming.

Building a similar CKAN integration? This README is a map of how it works and where things live. The exhaustive field list, DCAT mappings, and vocabulary architecture are not repeated here — they are generated from the schema and documented in docs/.

How it works

Everything hangs off a standard CKAN plugin, ckanext/marinerg/plugin.py, which implements a handful of CKAN interfaces:

Two other pieces are registered through CKAN config rather than a plugin interface:

• DCAT-AP 3.0 export — MarinergDCATProfile (profile.py) stacks on euro_dcat_ap_3 via ckanext.dcat.rdf.profiles, adding /catalog.{rdf,ttl,jsonld} and per-dataset serialisations.
• Keycloak OIDC — login and role mapping in oidc_plugin.py / oidc_roles.py / oidc_redirect.py.
• A second small plugin, marinerg_metrics (metrics.py), exposes a Prometheus metrics endpoint.

The metadata model is defined once

schema/marinerg_dataset.linkml.yaml (LinkML) is the single source of truth for the marine metadata fields and their controlled vocabularies. From it, scripts/generate_registry.py produces the controlled-vocabulary registry (ckanext/marinerg/registry/controlled-metadata.yaml) that the validators enforce and the RDF export cites. Edit the schema, then regenerate — never hand-edit the registry. tox -e registry (and CI) fails if the two drift.

Where things live

Run it locally

cd infra
docker compose -f compose.yml -f compose.local.yml up -d    # or: podman compose ...
# CKAN → http://localhost:5001   (login: ckan_admin / ckan_admin)

compose.local.yml mounts this repo into the container, so Python and template edits are picked up live — restart the ckan service to reload cached Jinja templates; only dependency changes need --build. The facility/equipment picker calls the Django facility service, expected on http://localhost:8000 (the image resolves it as http://host.docker.internal:8000).

Develop

python -m venv .venv && .venv/bin/pip install -e ".[test]"
.venv/bin/python -m pytest tests/ -v     # tests stub ckan.plugins.toolkit — run under plain pytest
tox -e format_check      # Black (format_apply to fix)
tox -e style             # flake8
tox -e type              # mypy
tox -e registry          # fails if the registry has drifted from the LinkML schema
python scripts/generate_registry.py   # regenerate the registry after a schema change

Vocabulary label caches (NERC NVS, SPDX, EuroSciVoc, GCMD, re3data) are pinned snapshots refreshed by the scripts/fetch_* scripts; run them manually and commit the reviewed diff.

Configuration

Marinerg config keys are baked into ckan.ini at image-build time (ckan config-tool in infra/Dockerfile); OIDC credentials and ckan.plugins are the deliberate runtime exceptions.

Release

Run a pipeline with RELEASE_FIELD = patch / minor / major. The tag it creates triggers the pipeline that publishes the pip package and the CKAN container image. Pin that image tag in the sibling ../infra repository, then run the infra deploy for the target environment.

Roadmap

NERC P01 parameters · SeaDataNet L05 device categories · DataCite export + DOI registration · OAI-PMH endpoint · MARINERG-i EOSC Virtual Organisation. Detail in docs/eosc-roadmap.md.

Licence

AGPL-3.0-or-later — see LICENSE. Copyright © Irish Centre for High-End Computing (ICHEC).