ckanext-marinerg

CKAN extension powering the MARINERG-i data catalogue — the data catalogue for the MARINERG-i e-infrastructure, a Distributed Research Infrastructure for Marine Renewable Energy (MRE) test facilities across the EU. On top of stock CKAN it adds a marine metadata schema, controlled-vocabulary validation, Zenodo/DOI prefill, a facility/equipment picker, DCAT-AP 3.0 + FAIR Signposting export, Keycloak OIDC login, and MARINERG-i theming.

Building a similar CKAN integration? This README is a map of how it works and where things live. The exhaustive field list, DCAT mappings, and vocabulary architecture are not repeated here — they are generated from the schema and documented in docs/.

How it works

Everything hangs off a standard CKAN plugin, ckanext/marinerg/plugin.py, which implements a handful of CKAN interfaces:

Two other pieces are registered through CKAN config rather than a plugin interface:

• DCAT-AP 3.0 export — MarinergDCATProfile (profile.py) stacks on euro_dcat_ap_3 via ckanext.dcat.rdf.profiles, adding /catalog.{rdf,ttl,jsonld} and per-dataset serialisations.
• Keycloak OIDC — login and role mapping in oidc_plugin.py / oidc_roles.py / oidc_redirect.py.
• A second small plugin, marinerg_metrics (metrics.py), exposes a Prometheus metrics endpoint.

The metadata model is a dependency, not a copy

The marine metadata fields and their controlled vocabularies are defined by marinerg-schema, which the validators and the RDF export read directly. Change a vocabulary there, release it, then move the pin here — the model has a second consumer (the facility client validates a bundle against it before minting a DOI), and both must be looking at the same vocabulary.

What stays in this repo is what is CKAN's own: schemas/dataset.yaml (the form — presets, snippets, help text) and the fetched authority labels under ckanext/marinerg/registry/ that the RDF export cites terms by. The form necessarily restates each enum's values for its UI, so scripts/check_schema_convergence.py holds it to the model on every merge request.

Where things live

Run it locally

cd infra
docker compose -f compose.yml -f compose.local.yml up -d    # or: podman compose ...
# CKAN → http://localhost:5001   (login: ckan_admin / ckan_admin)

compose.local.yml mounts this repo into the container, so Python and template edits are picked up live — restart the ckan service to reload cached Jinja templates; only dependency changes need --build. The facility/equipment picker calls the Django facility service, expected on http://localhost:8000 (the image resolves it as http://host.docker.internal:8000).

compose.local.yml also points the Zenodo prefill at a zenodo-mock on http://localhost:8766, so no local run reaches production Zenodo. Set CKAN__MARINERG__ZENODO_API_URL to override; it is baked at build time, so changing it needs --build.

Develop

uv sync --group dev
uv run pytest                       # tests stub ckan.plugins.toolkit — no CKAN install needed
uv run ruff check ckanext tests     # lint
uv run ruff format ckanext tests    # format (--check in CI)
uv run mypy ckanext                 # types
uv run python scripts/check_schema_convergence.py   # form vs. the metadata model

Ruff targets Python 3.10 — CKAN 2.11's runtime — even though the dev toolchain runs 3.14; see the comment in pyproject.toml before changing it.

Vocabulary label caches (NERC NVS, SPDX, EuroSciVoc, GCMD, re3data) are pinned snapshots refreshed by the scripts/fetch_* scripts; run them manually and commit the reviewed diff.

Configuration

Marinerg config keys are baked into ckan.ini at image-build time (ckan config-tool in infra/Dockerfile); OIDC credentials and ckan.plugins are the deliberate runtime exceptions, along with marinerg.portal_base_url.

Branding uses CKAN's own config rather than template overrides, so it stays editable at /ckan-admin/config without a rebuild: ckan.site_title, ckan.site_logo, ckan.favicon.

Theming

The theme is an additive stylesheet — assets/css/marinerg-theme.css, loaded after CKAN's compiled main.css via {% block styles %} in templates/base.html. Three rules keep it maintainable; all three were learned by breaking them:

1. Never restate what CKAN already styles. CKAN's .masthead .main-navbar ul li a already sets masthead link colour, padding and hover. Overriding it needed !important, which then had to be defended forever. There is no !important in this file — keep it that way.
2. Match CKAN's markup, don't fight it. Template overrides emit exactly what the block they replace emits. Adding Bootstrap's nav-item/nav-link to nav items build_nav_main renders bare pulled in foreign padding and colour, misaligning them by 11px.
3. Brand via config where CKAN offers it (logo, favicon, title above), CSS only for what it does not.

Why not a compiled SCSS theme (ckan.theme)? It replaces css/main wholesale, so we would own the Bootstrap + CKAN compile. Bootstrap's variables are overridable (!default), but CKAN's own 25 theme variables in public/base/scss/_variables.scss — $mastheadBackgroundColor and friends, the ones worth changing — carry no !default and are imported unconditionally by _ckan.scss. Overriding them means forking CKAN's import graph, which drifts silently on every upgrade. The additive layer is ~150 lines and costs nothing at upgrade time.

Colour choices are measured, not eyeballed: render a page headless and sample it, rather than trusting a summary statistic — a low standard deviation can still be a background you can plainly see.

Release

Run a pipeline with RELEASE_FIELD = patch / minor / major. The tag it creates triggers the pipeline that publishes the pip package and the CKAN container image. Pin that image tag in the sibling ../infra repository, then run the infra deploy for the target environment.

Roadmap

NERC P01 parameters · SeaDataNet L05 device categories · DataCite export + DOI registration · OAI-PMH endpoint · MARINERG-i EOSC Virtual Organisation. Detail in docs/eosc-roadmap.md.

Licence

AGPL-3.0-or-later — see LICENSE. Copyright © Irish Centre for High-End Computing (ICHEC).