Metadata-Version: 2.4
Name: ckanext-marinerg
Version: 0.1.1
Summary: CKAN extension for the MARINERG-i data catalogue: marine metadata schema, Zenodo prefill, facility/equipment picker, and DCAT-AP export.
Author-email: Irish Centre for High End Computing <james.grogan@ichec.ie>
License: AGPL-3.0-or-later
Project-URL: Repository, https://git.ichec.ie/marinerg-i/ckanext-marinerg
Project-URL: Homepage, https://git.ichec.ie/marinerg-i/ckanext-marinerg
Keywords: CKAN,Marine Renewable Energy,Data Catalogue,EOSC,DCAT
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Paste
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: ckanext-scheming
Requires-Dist: PyYAML
Requires-Dist: requests
Requires-Dist: pydantic>=2.0
Requires-Dist: prometheus-client>=0.20

# ckanext-marinerg

CKAN extension for the MARINERG-i data catalogue. Part of the MARINERG-i e-infrastructure — a Distributed Research Infrastructure supporting Marine Renewable Energy (MRE) test facilities across the EU.

Companion to the [data-access-service](https://git.ichec.ie/marinerg-i/data-access-service) repo, which contains the Docker compose stack, Django facility service, and planning documents.

---

## Features

### Marine metadata schema

Custom CKAN scheming fields on top of the standard dataset form:

| Field | Description |
|---|---|
| `zenodo_doi` | DOI entry with "Prefill from Zenodo" button |
| `publication_date` | `YYYY-MM-DD` |
| `visibility` | `public` / `private` / `restricted` — controlled vocab |
| `access_scope` | `open` / `org_members` / `on_request` — controlled vocab |
| `data_mode` | `Raw` / `Provisional` / `Delayed-mode` / `Mixed` — MARINERG-i vocab |
| `source_kind` | `zenodo` / `datacite` / `manual` / `synthetic` — MARINERG-i vocab |
| `feature_type` | CF DSG feature type — MARINERG-i SKOS vocab |
| `processing_level` | EMODnet EPL level (L0–L4) — NERC NVS EPL vocab |
| `coordinate_reference_system` | EPSG code or `CUSTOM` — NERC NVS L10 vocab |
| `site` | Named test site / location |
| `time_coverage_start` / `time_coverage_end` | Data collection period |
| `facility_ref` | Stable facility PID — ROR URI preferred, portal URL fallback |
| `facility_name` | Display name of selected facility |
| `equipment_name` | Name of selected equipment |

### Zenodo prefill

Server-side CKAN action (`marinerg_zenodo_prefill`) triggered by DOI entry. Fetches DataCite/InvenioRDM metadata from Zenodo using an optional API key held in CKAN server config and pre-populates the creation form. Handles both the current InvenioRDM format and the legacy Zenodo API format.

### Facility/equipment picker

Two-dropdown form widget that calls the Django facility service to populate contextual selections. Resolves each facility to a stable PID (ROR URI preferred; portal URL fallback) — internal Django API URLs are never written to dataset records.

### Controlled vocabulary validators

All controlled fields are validated at submission time against `registry/controlled-metadata.yaml`, which is auto-generated from the LinkML schema (`schema/marinerg_dataset.linkml.yaml`). Values not in the registry are rejected. Each code has a corresponding concept URI emitted in RDF export.

Vocabulary sources:
- `processing_level` → NERC NVS EPL (cached in `registry/nerc_cache/EPL.yaml`)
- `coordinate_reference_system` → NERC NVS L10 (cached in `registry/nerc_cache/L10.yaml`)
- `feature_type`, `access_scope`, `data_mode`, `source_kind`, `visibility` → MARINERG-i SKOS vocab
- `euroscivoc` → EuroSciVoc (EU Science Vocabulary; cached in `registry/euroscivoc_cache.yaml`)
- `gcmd_keyword` → NASA GCMD Science Keywords (cached in `registry/gcmd_keyword_cache.yaml`)
- `source_repository` → re3data registry (cached in `registry/source_repository_cache.yaml`)
- `license_id` → SPDX 3.x (cached in `registry/spdx_cache.yaml`; served via `licenses_group_url`)

The EOSC-vocabulary label caches are refreshed from their authorities with
`python scripts/fetch_eosc_vocabs.py` (URIs stay single-sourced in the LinkML schema).

### DCAT-AP 3.0 RDF export

`ckanext-dcat` is installed; `MarinergDCATProfile` stacks on `euro_dcat_ap_3`. Custom mappings:

- `visibility` → `dct:accessRights` (EU Publications Office vocab)
- `facility_ref` / `facility_name` → `dct:publisher` (ROR URI as `foaf:homepage`)
- `author` → `dct:creator`; `author_email` → `dcat:contactPoint` (restricted datasets only)
- `zenodo_doi` → `dct:identifier` + `adms:identifier` (the DOI is the dataset's sole identity; no `owl:sameAs`)
- `publication_date` → `dct:issued`
- `feature_type` → `dcat:theme` (MARINERG-i concept URI)
- `euroscivoc` → `dcat:theme` (EuroSciVoc concept URIs); `gcmd_keyword` → `dct:subject` (GCMD concept URIs)
- `source_repository` → `dct:source` (re3data repository URI)
- `processing_level` → `dqv:hasQualityAnnotation` with NERC EPL concept URI
- `coordinate_reference_system` → `dct:conformsTo` (NERC L10 URI or Literal)
- `site` → `dct:spatial`; `time_coverage_start/end` → `dct:temporal`

Endpoints: `/catalog.rdf`, `/catalog.ttl`, `/catalog.n3`, `/catalog.jsonld` and per-dataset equivalents.

### FAIR Signposting

Dataset landing pages (`/dataset/<name>`) carry a typed HTTP `Link` header (FAIR
Signposting Profile), added by an `IBlueprint` app-wide `after_request` hook
(`signposting.py`):

- `cite-as` → the DOI (canonical citation)
- `describedby` → the `.ttl` / `.jsonld` / `.rdf` DCAT serialisations (with media types)
- `type` → `dcat:Dataset` + `schema.org/Dataset`
- `author` → ORCiD URI (when recorded), `license` → licence URL, `item` → each distribution

Machine-navigable without OAI-PMH or SPARQL; assessed by F-UJI and expected by EOSC
harvesters. Ref: `../data-access-service/docs/eosc-roadmap.md` §2.9.

### CKAN theming

Custom header (MARINERG-i logo), colour scheme (`--marinerg-primary: #003964`), favicon, background imagery.

---

## Planned

- **Keycloak SSO** — `ckanext-oidc-pkce` against ICHEC Keycloak; JIT user creation; group membership from JWT claims
- **NERC P01 parameters** — discipline-filtered allowlist; instrument/parameter linking
- **SeaDataNet device categories (L05)** — upgrade `equipment_name` from free text to controlled picker
- **DataCite export** — same schema and vocab registry; DOI registration
- **OAI-PMH endpoint** — for EOSC/B2FIND harvesting
- **MARINERG-i VO** — EOSC Virtual Organisation for consortium-level access policies

---

## Local development

This repo is a sibling of `data-access-service/`. The CKAN dev stack lives there:

```sh
cd ../data-access-service/infra/ckan
docker compose -f compose.yml -f compose.local.yml up -d --build
# CKAN at http://localhost:5001  (port 5000 is owned by Docker Desktop's gvproxy)
# Login: ckan_admin / ckan_admin
```

`compose.local.yml` mounts this repo into the container at `/srv/app/src/ckanext-marinerg`. Python and template changes are live immediately. Only dependency changes need a rebuild.

### Facility service

The facility/equipment picker calls the Django facility service. For local dev it should be running on port 8000. The CKAN image has `http://host.docker.internal:8000` baked in at build time — Docker resolves this to the Mac host from inside the container.

### Running tests

```sh
python -m venv .venv
.venv/bin/pip install -e ".[test]"
.venv/bin/python -m pytest tests/ -v
```

All tests run without a CKAN installation — `conftest.py` stubs `ckan.plugins.toolkit` so pure action/model/validator logic can be tested directly.

### Linting and formatting

```sh
tox -e format_check   # check (CI)
tox -e format_apply   # apply
tox -e style          # flake8
tox -e type           # mypy
tox -e registry       # check controlled-metadata.yaml is in sync with the LinkML schema
```

---

## Vocabulary cache scripts

Run these manually (or on a schedule in CI) to update pinned snapshots. Commit the diff after reviewing.

```sh
# NERC NVS collections (EPL processing levels, L10 CRS)
python scripts/fetch_nerc_vocabs.py

# SPDX license list
python scripts/fetch_spdx_licenses.py

# Regenerate controlled-metadata.yaml from the LinkML schema (also run by `tox -e registry`)
python scripts/generate_registry.py
```

---

## CKAN config keys

Baked into `ckan.ini` at image build time via `RUN ckan config-tool` in `data-access-service/infra/ckan/Dockerfile`. Do not rely on runtime env vars — `prerun.py` only writes `ckan.plugins`.

| Key | Default | Purpose |
|---|---|---|
| `scheming.dataset_schemas` | — | Path to the marine dataset schema |
| `ckanext.dcat.rdf.profiles` | `euro_dcat_ap_3 marinerg_dcat` | RDF profile stack |
| `marinerg.zenodo_api_key` | `""` | Zenodo API key for prefill (optional) |
| `marinerg.facility_api_url` | `http://host.docker.internal:8000` | Django facility service base URL |
| `marinerg.facility_api_token` | `""` | Token for authenticated facility API access |
| `marinerg.portal_base_url` | `http://localhost:5001` | Fallback facility PID base when no ROR ID |

---

## Licence

Copyright ICHEC. GNU AGPL v3 or later.
