Metadata-Version: 2.2
Name: ckanext-zenodo
Version: 0.1.19
Summary: Publishes large or non-tabular dataset files to Zenodo (sandbox or production) instead of CKAN storage, mapping FDF metadata into the Zenodo deposition.
Author-email: ICS <groupe-info-ics@igbmc.fr>
License: AGPL-3.0-or-later
Keywords: CKAN,data,zenodo,doi
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests
Provides-Extra: test
Requires-Dist: pytest==7.4.4; extra == "test"
Requires-Dist: pytest-cov==4.1.0; extra == "test"

![CKAN](https://img.shields.io/badge/CKAN-2.12.0-orange)
![License](https://img.shields.io/badge/license-AGPL--3.0-blue)

# ckanext-zenodo

Publishes large or non-tabular dataset files to [Zenodo](https://zenodo.org/)
(sandbox during development, production later) instead of CKAN storage,
mapping the dataset's FDF metadata into the Zenodo deposition. The CKAN
dataset stays published in the FAIR3R catalogue (full FDF metadata,
searchable) with a link to the Zenodo DOI for the actual data, instead of
carrying the file itself.

## Status

Work in progress, built in staged merge requests. See the implementation
plan for the full design (file flow, data model, metadata mapping,
anti-abuse hardening, staged delivery order) - ask in the project chat for
the current plan reference if you don't have it to hand.

Routing rule (once implemented): a file goes to Zenodo instead of CKAN
storage when it's larger than `ckan.max_resource_size`, or its format isn't
one `ckanext.xloader.formats` can read - or whenever the user explicitly
asks for Zenodo regardless of size/format.

## Relationship to ckanext-fair3r

Not a declared package dependency (not in `pyproject.toml`), but FAIR3R is
this plugin's reason to exist, at two different levels:

- **Soft Python dependency** - `lib/publish_guard.py` imports
  `ckanext.fair3r.lib.fdf.datacite_converter.DataCiteConverter` inside a
  `try`/`except ImportError` to map a dataset's FDF metadata into the
  Zenodo deposition's description/creators/keywords/etc. Same pattern
  `ckanext-citations`' `helpers.py` already uses for
  `completeness_for_package`. Without `ckanext-fair3r` installed (or on a
  dataset with no `fdf_output_json` extra), a deposition still gets
  published - just with minimal metadata (title and the submitting user's
  name only), never a hard failure.
- **Operational dependency, not a code one** - the actual trigger for
  uploading to this plugin's `/zenodo/upload/<package>` endpoint lives
  entirely in `ckanext-fair3r`'s own `resource-upload.js` (the
  size/format routing decision and the "force Zenodo" checkbox). Without
  `ckanext-fair3r` installed *and enabled*, nothing in the normal CKAN UI
  ever calls this plugin's upload endpoint at all - it would need a
  hand-crafted request. See the implementation plan's staged delivery
  order for exactly where that wiring lands.

In short: install this plugin on its own and it runs (migrations apply,
the blueprint responds, tests pass), but it's only actually useful
alongside a FAIR3R deployment that also runs `ckanext-fair3r`.

## Sandbox smoke test findings (Stage 8)

Confirmed empirically against the real `sandbox.zenodo.org` (the current,
InvenioRDM-based Zenodo) - two things the implementation plan flagged as
unverified assumptions, now settled:

- **`metadata.creators` is required to publish**, even though updating a
  deposition's metadata without it is accepted - omitting it makes
  `publish_deposition` fail every time with a 400. `metadata_mapper.py`'s
  `datacite_to_zenodo` now always sets `creators`, falling back to the
  submitter's name when the DataCite metadata has none (e.g. no
  `ckanext-fair3r`, or no FDF metadata yet).
- **Setting `metadata.communities` does not associate the record with
  that community by itself** - it only opens a pending
  `community-inclusion` request against the record, which stays
  unaccepted (and invisible under the community) until explicitly
  accepted, even for the account that owns the community. `publish_guard.py`
  now does this automatically right after a successful publish
  (`_accept_pending_community_inclusion`) - best-effort: a failure here
  is logged but never undoes the publish (the DOI is already minted by
  that point).
- The description HTML sanitizer runs **on write**, not just on display,
  and its real allow-list is now documented directly above
  `build_zenodo_description_html` in `metadata_mapper.py` (confirmed by
  round-tripping a probe deposition through every plausible tag and
  reading back what Zenodo actually stored).

## When a deposition actually gets published to Zenodo

Same trigger condition as ckanext-doi's own `after_dataset_update`
(already established in this project, for this exact question, on this
exact CKAN dataset): the dataset must be **active AND not private**,
not just active. Minting a Zenodo DOI is a one-way action, same as
DataCite's own - a dataset that's active but still private never gets
one yet. The hook fires on every dataset update, not just once right
after creation, so a dataset created active+private and made public
only later still gets published the next time it's updated (even just
toggling that flag) - no catch-up job needed. One consequence: by the
time a deposition is actually published, the dataset is always already
public, so Zenodo's `access_right` is always `'open'` - there's no way
to end up with a stale `'restricted'` record whose CKAN dataset later
went public without Zenodo ever being told.

## How the published record shows up in CKAN

`publish_guard.py` creates one genuine CKAN **Link resource per file**
actually uploaded to Zenodo (`url` = the published record's landing
page, same url for every file in that deposition - no file upload into
CKAN itself), named after each file's real filename - not a bespoke
display panel bolted on elsewhere, and not one generic entry for the
whole deposition either. A real resource needs no template override to
show up in the normal "Data and Resources" list, and - more importantly
- gets picked up by anything that already iterates `pkg.resources`
(ckanext-fair3r's DCAT/schema.org/Croissant export included), which a
panel outside the resource list never would be. Idempotent by
(name, url) rather than a new DB column: a retried publish job that
already created some or all of these resources on an earlier,
interrupted attempt won't create duplicates.

A minimal notice ("...is being archived on Zenodo - reload this page in
a moment...") shows instead, in the same "Data and Resources" area,
for the (usually short) gap between the file reaching Zenodo and the
publish job above actually running - see `lib/helpers.py`'s
`zenodo_pending_deposition_for_package()`.

## Tests

```shell
docker exec -u ckan -it ckan-app pytest --ckan-ini=/plugins/ckanext-zenodo/test.ini /plugins/ckanext-zenodo/ckanext/zenodo/tests
```

## License

Released under the GNU Affero General Public License (AGPL) v3.0. See the
file `LICENSE` for details.
