Metadata-Version: 2.4
Name: sconext-ckan
Version: 0.1.1
Summary: CKAN catalog, data service and policy evaluator for the SingularConnector
Home-page: https://www.singularlogic.eu/
Author-email: Iason Sotiropoulos <isotiropoulos@singularlogic.eu>
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: pydantic>=2.7.0
Requires-Dist: scontoolkit>=0.0.3b1
Requires-Dist: ckanapi
Requires-Dist: httpx>=0.27.0
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-asyncio; extra == "test"
Dynamic: home-page

# sconext-ckan

One extension, three faces of a CKAN-hosted asset. Install it and the
SingularConnector has a complete dataspace path for every dataset in your CKAN:
catalogued, negotiable, transferable, and enforced.

| interface | class | what it does |
|---|---|---|
| `ICatalogService` | `CKANCatalogService` | how to **find** the assets |
| `IDataService` | `CKANDataService` | how to **fetch** the data |
| `IPolicyEvaluator` | `CKANPolicyEvaluator` | how to **apply a duty** to it |

```bash
pip install sconext-ckan

export sconext.ckan.url=https://data.example.eu
export sconext.ckan.admin_key=...                       # optional, for private datasets
export sconnector.participant_id=urn:example:city-of-athens
export sconnector.dsp.endpoint=https://connector.example.eu/2025-1
```

That is the whole installation. All three classes are declared in the single
`sconext` entry-point group; `ExtensionRegistry` registers each instance under
every `scontoolkit.interfaces` base in its MRO, so the connector finds all
three in one discovery pass and needs no change.

## Configuration

| variable | required | meaning |
|---|---|---|
| `sconext.ckan.url` | yes | CKAN base URL |
| `sconext.ckan.admin_key` | no | CKAN API key; without it only public datasets are visible |
| `sconext.ckan.timeout` | no | seconds, default 30 |
| `sconnector.participant_id` | yes | `dspace:participantId` for the root Catalog |
| `sconnector.dsp.endpoint` | yes | control-plane base **including the version segment** — this becomes `DataService.endpointURL` |
| `sconext.ckan.catalog_id` | no | root Catalog `@id`; derived from the participant id otherwise |
| `sconext.ckan.data_service_id` | no | `@id` of the advertised DataService |

`sconnector.dsp.endpoint` is the **control** plane, not the data plane. DSP:
*"A DataService.endpointURL property contains the URL of the service the
Contract Negotiation endpoints extend."* Pointing it at `/data` is the
intuitive mistake and it sends consumers to an endpoint that will refuse them,
because they have no Agreement yet.

## Which datasets appear

A CKAN dataset appears in the DSP catalog only once an **offer has been
exposed** for it, because DSP requires every Dataset to carry at least one
`hasPolicy` Offer. Expose offers through the connector's management API. The
operator-facing `list_assets` deliberately shows *every* CKAN dataset, exposed
or not — you have to see one in order to publish an offer for it.

## What the data service reads

Per asset it picks one resource: the one named in `params["resource_id"]`, else
the first DataStore-backed resource, else the first parseable one.

* **DataStore-backed** → `datastore_search`, paged at 1000 rows. Field
  projection and filters push to source.
* **Otherwise** → the resource `url` is downloaded and parsed. CSV, TSV, JSON
  and GeoJSON only.

A dataset whose only resources are in another format is **refused by name**,
not returned empty. `read()` on a PDF-only dataset raises `UnsupportedAction`
naming the formats found, because a transfer that returns nothing must not look
like a transfer of nothing.

Read-only. `write`, `write_setpoint`, `derive`, `train`, `evaluate` and
`transfer` raise `UnsupportedAction` with a reason. `supported_actions()`
advertises only what it will actually do, so an agreement over an action this
service refuses is declined at negotiation rather than failing at request time.

**Policy-blind**, and not as a style preference: nothing here anonymises,
aggregates, suppresses or throttles. A bound enforced inside a policy-blind
plug-in is a bound the PEP cannot observe, so it must not be reported as
enforced. `stream()` in particular does not pace itself — the connector paces
subscriptions on its own side where the decision is auditable.

## What the evaluator does

It subclasses `BaseUedsEvaluator`, so all 18 UEDS egress duties are inherited
and `supported_duties()` reports all 18. Two overrides:

**Record identity.** CKAN DataStore rows key on `_id` and carry no `meterId`.
With the base class's identity fields, every row counts as identity `None`, the
distinct count collapses to zero, and *every* `ueds:meteringPointCount` floor
suppresses the whole payload — over-restrictive, and invisibly wrong. There is
a test that fails without this.

**Provenance.** `attach_provenance` adds the upstream CKAN licence, publisher
and landing page. An open-data publisher sets terms upstream of the dataspace;
a consumer that receives the records without them cannot honour them, and no
generic evaluator can supply them.

Everything else — k-anonymity (including the absent-quasi-identifier guard),
aggregation, differential privacy, pseudonymisation, coarsening, embargo,
projection, the cohort guard itself — is inherited unchanged.

## Tests

```bash
pip install -e '.[test]'
python -m pytest tests/ -q          # 31 passed, no network, no credentials
```

CKAN is stubbed in `tests/conftest.py`. The previous suite hardcoded a live
CKAN admin API key and pointed at a real instance, so it could not run offline
or in CI. **If you have not already, rotate that key** — it is in the git
history of this folder even though it is gone from the working tree.

## One connector change still needed

The catalog and data service work against the connector as it stands today.
`CKANPolicyEvaluator` registers and is discoverable, but the connector's PEP
still applies duties through its own `pep/transformers.py` rather than
consulting `IPolicyEvaluator`. Until `apply_transforms` dispatches through
`DUTY_METHODS` to the evaluator that `serves()` the asset, this class is
installed and unused — the two CKAN-specific overrides above will not take
effect, which for the cohort floor means the connector suppresses payloads it
should deliver.
