ckanext-marinerg

CKAN extension for the MARINERG-i data catalogue. Part of the MARINERG-i e-infrastructure — a Distributed Research Infrastructure supporting Marine Renewable Energy (MRE) test facilities across the EU.

Companion to the data-access-service repo, which contains the Docker compose stack, Django facility service, and planning documents.

---

Features

Marine metadata schema

Custom CKAN scheming fields on top of the standard dataset form:

Field	Description
zenodo_doi	DOI entry with "Prefill from Zenodo" button
publication_date	YYYY-MM-DD
visibility	public / private / restricted — controlled vocab
access_scope	open / org_members / on_request — controlled vocab
data_mode	Raw / Provisional / Delayed-mode / Mixed — MARINERG-i vocab
source_kind	zenodo / datacite / manual / synthetic — MARINERG-i vocab
feature_type	CF DSG feature type — MARINERG-i SKOS vocab
processing_level	EMODnet EPL level (L0–L4) — NERC NVS EPL vocab
coordinate_reference_system	EPSG code or CUSTOM — NERC NVS L10 vocab
site	Named test site / location
time_coverage_start / time_coverage_end	Data collection period
facility_ref	Stable facility PID — ROR URI preferred, portal URL fallback
facility_name	Display name of selected facility
equipment_name	Name of selected equipment

Zenodo prefill

Server-side CKAN action (marinerg_zenodo_prefill) triggered by DOI entry. Fetches DataCite/InvenioRDM metadata from Zenodo using an optional API key held in CKAN server config and pre-populates the creation form. Handles both the current InvenioRDM format and the legacy Zenodo API format.

Facility/equipment picker

Two-dropdown form widget that calls the Django facility service to populate contextual selections. Resolves each facility to a stable PID (ROR URI preferred; portal URL fallback) — internal Django API URLs are never written to dataset records.

Controlled vocabulary validators

All controlled fields are validated at submission time against registry/controlled-metadata.yaml, which is auto-generated from the LinkML schema (schema/marinerg_dataset.linkml.yaml). Values not in the registry are rejected. Each code has a corresponding concept URI emitted in RDF export.

Vocabulary sources:

• processing_level → NERC NVS EPL (cached in registry/nerc_cache/EPL.yaml)
• coordinate_reference_system → NERC NVS L10 (cached in registry/nerc_cache/L10.yaml)
• feature_type, access_scope, data_mode, source_kind, visibility → MARINERG-i SKOS vocab
• euroscivoc → EuroSciVoc (EU Science Vocabulary; cached in registry/euroscivoc_cache.yaml)
• gcmd_keyword → NASA GCMD Science Keywords (cached in registry/gcmd_keyword_cache.yaml)
• source_repository → re3data registry (cached in registry/source_repository_cache.yaml)
• license_id → SPDX 3.x (cached in registry/spdx_cache.yaml; served via licenses_group_url)

The EOSC-vocabulary label caches are refreshed from their authorities with python scripts/fetch_eosc_vocabs.py (URIs stay single-sourced in the LinkML schema).

DCAT-AP 3.0 RDF export

ckanext-dcat is installed; MarinergDCATProfile stacks on euro_dcat_ap_3. Custom mappings:

• visibility → dct:accessRights (EU Publications Office vocab)
• facility_ref / facility_name → dct:publisher (ROR URI as foaf:homepage)
• author → dct:creator; author_email → dcat:contactPoint (restricted datasets only)
• zenodo_doi → dct:identifier + adms:identifier (the DOI is the dataset's sole identity; no owl:sameAs)
• publication_date → dct:issued
• feature_type → dcat:theme (MARINERG-i concept URI)
• euroscivoc → dcat:theme (EuroSciVoc concept URIs); gcmd_keyword → dct:subject (GCMD concept URIs)
• source_repository → dct:source (re3data repository URI)
• processing_level → dqv:hasQualityAnnotation with NERC EPL concept URI
• coordinate_reference_system → dct:conformsTo (NERC L10 URI or Literal)
• site → dct:spatial; time_coverage_start/end → dct:temporal

Endpoints: /catalog.rdf, /catalog.ttl, /catalog.n3, /catalog.jsonld and per-dataset equivalents.

FAIR Signposting

Dataset landing pages (/dataset/<name>) carry a typed HTTP Link header (FAIR Signposting Profile), added by an IBlueprint app-wide after_request hook (signposting.py):

• cite-as → the DOI (canonical citation)
• describedby → the .ttl / .jsonld / .rdf DCAT serialisations (with media types)
• type → dcat:Dataset + schema.org/Dataset
• author → ORCiD URI (when recorded), license → licence URL, item → each distribution

Machine-navigable without OAI-PMH or SPARQL; assessed by F-UJI and expected by EOSC harvesters. Ref: ../data-access-service/docs/eosc-roadmap.md §2.9.

CKAN theming

Custom header (MARINERG-i logo), colour scheme (--marinerg-primary: #003964), favicon, background imagery.

---

Planned

• Keycloak SSO — ckanext-oidc-pkce against ICHEC Keycloak; JIT user creation; group membership from JWT claims
• NERC P01 parameters — discipline-filtered allowlist; instrument/parameter linking
• SeaDataNet device categories (L05) — upgrade equipment_name from free text to controlled picker
• DataCite export — same schema and vocab registry; DOI registration
• OAI-PMH endpoint — for EOSC/B2FIND harvesting
• MARINERG-i VO — EOSC Virtual Organisation for consortium-level access policies

---

Local development

This repo is a sibling of data-access-service/. The CKAN dev stack lives there:

cd ../data-access-service/infra/ckan
docker compose -f compose.yml -f compose.local.yml up -d --build
# CKAN at http://localhost:5001  (port 5000 is owned by Docker Desktop's gvproxy)
# Login: ckan_admin / ckan_admin

compose.local.yml mounts this repo into the container at /srv/app/src/ckanext-marinerg. Python and template changes are live immediately. Only dependency changes need a rebuild.

Facility service

The facility/equipment picker calls the Django facility service. For local dev it should be running on port 8000. The CKAN image has http://host.docker.internal:8000 baked in at build time — Docker resolves this to the Mac host from inside the container.

Running tests

python -m venv .venv
.venv/bin/pip install -e ".[test]"
.venv/bin/python -m pytest tests/ -v

All tests run without a CKAN installation — conftest.py stubs ckan.plugins.toolkit so pure action/model/validator logic can be tested directly.

Linting and formatting

tox -e format_check   # check (CI)
tox -e format_apply   # apply
tox -e style          # flake8
tox -e type           # mypy
tox -e registry       # check controlled-metadata.yaml is in sync with the LinkML schema

---

Vocabulary cache scripts

Run these manually (or on a schedule in CI) to update pinned snapshots. Commit the diff after reviewing.

# NERC NVS collections (EPL processing levels, L10 CRS)
python scripts/fetch_nerc_vocabs.py

# SPDX license list
python scripts/fetch_spdx_licenses.py

# Regenerate controlled-metadata.yaml from the LinkML schema (also run by `tox -e registry`)
python scripts/generate_registry.py

---

CKAN config keys

Baked into ckan.ini at image build time via RUN ckan config-tool in data-access-service/infra/ckan/Dockerfile. Do not rely on runtime env vars — prerun.py only writes ckan.plugins.

Key	Default	Purpose
scheming.dataset_schemas	—	Path to the marine dataset schema
ckanext.dcat.rdf.profiles	euro_dcat_ap_3 marinerg_dcat	RDF profile stack
marinerg.zenodo_api_key	""	Zenodo API key for prefill (optional)
marinerg.facility_api_url	http://host.docker.internal:8000	Django facility service base URL
marinerg.facility_api_token	""	Token for authenticated facility API access
marinerg.portal_base_url	http://localhost:5001	Fallback facility PID base when no ROR ID

---

Licence

Copyright ICHEC. GNU AGPL v3 or later.