Metadata-Version: 2.4
Name: dapla-toolbelt-datafangst
Version: 0.0.2
Summary: Dapla Toolbelt Datafangst
License-Expression: MIT
License-File: LICENSE
Author: Contact Person
Author-email: rupinder.kaur@ssb.no
Maintainer: Statistics Norway, Data collection platform Department (722)
Requires-Python: >=3.12
Classifier: Development Status :: 5 - Production/Stable
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: click (>=8.0.1)
Requires-Dist: dapla-auth-client (>=1.2.1)
Requires-Dist: pydantic (>=2.11)
Requires-Dist: python-dateutil (>=2.8.2)
Requires-Dist: urllib3 (>=2.1.0,<3.0.0)
Project-URL: Changelog, https://github.com/statisticsnorway/dapla-toolbelt-datafangst/releases
Project-URL: Documentation, https://statisticsnorway.github.io/dapla-toolbelt-datafangst
Project-URL: Homepage, https://github.com/statisticsnorway/dapla-toolbelt-datafangst
Project-URL: Repository, https://github.com/statisticsnorway/dapla-toolbelt-datafangst
Description-Content-Type: text/markdown

# Dapla Toolbelt Datafangst

[![PyPI](https://img.shields.io/pypi/v/dapla-toolbelt-datafangst.svg)][pypi status]
[![Status](https://img.shields.io/pypi/status/dapla-toolbelt-datafangst.svg)][pypi status]
[![Python Version](https://img.shields.io/pypi/pyversions/dapla-toolbelt-datafangst)][pypi status]
[![License](https://img.shields.io/pypi/l/dapla-toolbelt-datafangst)][license]

[![Documentation](https://github.com/statisticsnorway/dapla-toolbelt-datafangst/actions/workflows/docs.yml/badge.svg)][documentation]
[![Tests](https://github.com/statisticsnorway/dapla-toolbelt-datafangst/actions/workflows/tests.yml/badge.svg)][tests]
[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=statisticsnorway_dapla-toolbelt-datafangst&metric=coverage)][sonarcov]
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=statisticsnorway_dapla-toolbelt-datafangst&metric=alert_status)][sonarquality]

[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)][pre-commit]
[![Black](https://img.shields.io/badge/code%20style-black-000000.svg)][black]
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Poetry](https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json)][poetry]

[pypi status]: https://pypi.org/project/dapla-toolbelt-datafangst/
[documentation]: https://statisticsnorway.github.io/dapla-toolbelt-datafangst
[tests]: https://github.com/statisticsnorway/dapla-toolbelt-datafangst/actions?workflow=Tests
[sonarcov]: https://sonarcloud.io/summary/overall?id=statisticsnorway_dapla-toolbelt-datafangst
[sonarquality]: https://sonarcloud.io/summary/overall?id=statisticsnorway_dapla-toolbelt-datafangst
[pre-commit]: https://github.com/pre-commit/pre-commit
[black]: https://github.com/psf/black
[poetry]: https://python-poetry.org/

Python client library for the **Kudoc** — the metadata registry for data collections at Statistics Norway (SSB). Use it from Jupyter notebooks or Python scripts to browse published collections and manage draft collections programmatically.

## Features

- **Browse published collections** — list, get by ID, search by name, search by Altinn questionnaire ID, and retrieve version history.
- **Manage draft collections** — create, read, update, delete, and publish drafts.
- **Automatic authentication** — tokens are fetched and refreshed transparently via [dapla-auth-client].
- **Environment-aware** — automatically resolves the correct API host for DEV, QA, and TEST environments.

## Requirements

- Python 3.12+
- A valid Dapla user account (for authentication)
- The `DAPLA_ENVIRONMENT` environment variable set when running on the Dapla platform

## Installation

```console
pip install dapla-toolbelt-datafangst
```

## Quick Start

### List all published collections

```python
import json
from dapla_toolbelt_datafangst import Collections

collections = Collections.list_collections()
print(json.dumps([c.to_dict() for c in collections], indent=2, default=str))
```

### Get a single collection by ID

```python
from dapla_toolbelt_datafangst import Collections

collection = Collections.get_collection(collection_id=42)
print(collection.name)
print(collection.to_json())
```

### Search collections by name

```python
from dapla_toolbelt_datafangst import Collections

results = Collections.search_by_name("Loennsstatistikk")
for c in results:
    print(f"{c.id}: {c.name}")
```

### Search collections by Altinn questionnaire ID

```python
from dapla_toolbelt_datafangst import Collections

results = Collections.search_by_altinn_id("RA-0329")
for c in results:
    print(f"{c.id}: {c.name}")
```

### Get all versions of a collection

```python
from dapla_toolbelt_datafangst import Collections

versions = Collections.get_versions(collection_id=42)
for v in versions:
    info = v.version_information
    print(f"Version {info.id} — status: {info.status}, updated: {info.last_updated_at}")
```

## Working with Drafts

### List your own drafts

```python
import json
from dapla_toolbelt_datafangst import Drafts

my_drafts = Drafts.list_mine()
print(json.dumps([d.to_dict() for d in my_drafts], indent=2, default=str))
```

### List drafts from all your teams

```python
from dapla_toolbelt_datafangst import Drafts

team_drafts = Drafts.list_team_drafts()
for d in team_drafts:
    print(f"{d.id}: {d.name} (team: {d.dapla_team.display_name})")
```

### Get a specific draft

```python
from dapla_toolbelt_datafangst import Drafts

draft = Drafts.get(draft_id=123)
print(draft.to_json())
```

### Create a new draft

```python
from dapla_toolbelt_datafangst import (
    CollectionInstrument,
    Drafts,
    RequestCollection,
    ToolConfig,
    ToolAltinn,
)
from dapla_toolbelt_datafangst._generated.kudoc_client.models.altinn_form import AltinnForm

draft = Drafts.create(
    RequestCollection(
        name="My New Collection",
        dapla_team_uniform_name="team-my-team",
        spec_categ_data=False,
        collection_instrument=[
            CollectionInstrument(
                name="Altinn skjema",
                tool=ToolConfig(
                    actual_instance=ToolAltinn(
                        tool_type="skjemaAltinn",
                        forms=[AltinnForm(id="RA-0329", prefill=False)],
                    )
                ),
            )
        ],
    )
)
print(f"Created draft with ID: {draft.id}")
```

### Update an existing draft

```python
from dapla_toolbelt_datafangst import Drafts, RequestCollection

existing = Drafts.get(draft_id=123)

updated = Drafts.update(
    draft_id=123,
    draft=RequestCollection(
        name="Updated Collection Name",
        dapla_team_uniform_name=existing.dapla_team.uniform_name,
        spec_categ_data=existing.spec_categ_data,
        collection_instrument=existing.collection_instrument,
    ),
)
print(f"Updated: {updated.name}")
```

### Publish a draft

```python
from dapla_toolbelt_datafangst import Drafts

published = Drafts.publish(draft_id=123)
print(f"Published collection ID: {published.id}")
```

### Delete a draft

```python
from dapla_toolbelt_datafangst import Drafts

Drafts.delete(draft_id=123)
```

## Data Models

### RequestCollection

Used when creating or updating a draft.

| Field | Type | Required | Description |
|---|---|---|---|
| `name` | `str` | Yes | Name of the collection |
| `dapla_team_uniform_name` | `str` | Yes | Dapla team uniform name (e.g. `team-my-team`) |
| `spec_categ_data` | `bool` | Yes | Whether the collection contains special category data |
| `collection_instrument` | `list[CollectionInstrument]` | Yes | List of collection instruments |
| `assignment_type` | `AssignmentType` | No | `Stat` (statistical) or `Marked` (commercial) |
| `relevant_docs` | `list[RelevantDoc]` | No | Links to relevant documents (DPIA, vedtak, etc.) |
| `reporting_unit_type` | `list[str]` | No | Reporting unit types |
| `statistics_register_id` | `str` | No | Statistics register identifier |
| `information_provider` | `list[str]` | No | Information provider codes (KLASS #712) |
| `valid_from` | `datetime` | No | Validity start date |
| `valid_until` | `datetime` | No | Validity end date |
| `sample_plan` | `SamplePlan` | No | Sampling plan details |
| `period_plan` | `PeriodPlan` | No | Period plan configuration |

### ResponseCollection

Returned from all read operations. Contains all `RequestCollection` fields plus:

| Field | Type | Description |
|---|---|---|
| `id` | `str` | Unique collection identifier |
| `dapla_team` | `Team` | Full team object (`uniform_name`, `display_name`, `section_name`) |
| `version_information` | `VersionInformation` | Version ID, status, timestamps, and author |

### CollectionInstrument

Describes a single instrument within a collection.

| Field | Type | Description |
|---|---|---|
| `id` | `str` | Unique identifier |
| `name` | `str` | Human-readable name |
| `instrument_source` | `str` | Information provider for this instrument |
| `instrument_frequency` | `str` | Frequency: `Engangs`, `Aar`, `Halvaar`, `Kvartal`, `Termin`, `Maaned`, `Uke`, `Dag`, `Kontinuerlig` |
| `ext_contact` | `list[InstrumentContact]` | External contact persons |
| `int_contact` | `list[InstrumentContact]` | Internal SSB contact persons |
| `tool` | `ToolConfig` | Collection tool configuration (one of `ToolAltinn`, `ToolBlaise`, or `ToolAPI`) |

### Tool Types

`ToolConfig` wraps one of three tool types:

**ToolAltinn** — Altinn form-based collection:
- `tool_type`: `"skjemaAltinn"`
- `forms`: list of `AltinnForm` (each with `id` and `prefill`)

**ToolBlaise** — Blaise interview-based collection:
- `tool_type`: `"blaise"`
- `name_acronym`, `survey_type` (`Panel` / `Tverrsnitt`), `interview_mode` (`CAPI`, `CATI`, `CAWI`, `PAPI`, `APP`), `volume`

**ToolAPI** — API-based collection:
- `tool_type`: `"api"`
- `api_name`, `description`, `api_url_test`, `api_url_prod`, `authentication_type` (`maskinporten`, `oauth2`, `api_key`, `basic`, `none`), `data_format` (`json`, `csv`, `xml`, `parquet`), and more

### Enums

| Enum | Values |
|---|---|
| `Status` | `DRAFT`, `PUBLISHED`, `ARCHIVED` |
| `AssignmentType` | `Stat`, `Marked` |
| `RelevantDocType` | `dpia`, `registermelding`, `vedtak`, `other_vedtak`, `delivery_agreement`, `cooperation_agreement`, `other_agreement` |

## Configuration

### Environment Variables

| Variable | Description | Default |
|---|---|---|
| `DAPLA_ENVIRONMENT` | Dapla lifecycle environment (`DEV`, `QA`, `TEST`) | Not set (uses localhost) |
| `KUDOC_HOST` | Manual override for the Kudoc API URL | `http://localhost:8080` |

When `DAPLA_ENVIRONMENT` is set, the library automatically resolves the correct API host:

| Environment | API Host |
|---|---|
| `DEV` | `https://dev-collection-gateway.intern.test.ssb.no/kudoc` |
| `QA` | `https://qa-collection-gateway.intern.test.ssb.no/kudoc` |
| `TEST` | `https://test-collection-gateway.intern.test.ssb.no/kudoc` |

> **Note:** PROD environment is not yet available.

### Authentication

Authentication is handled automatically via [dapla-auth-client]. The library fetches a personal access token with scopes `all_groups` and `current_group` for the `kudoc` audience. Tokens are refreshed on every API call.

If you get an `Unauthorized` error, try logging out and back in to your Dapla session.

## Error Handling

All API errors are wrapped in `DaplaClientError` (also aliased as `KudocClientError`) with user-friendly messages:

```python
from dapla_toolbelt_datafangst import Collections, DaplaClientError

try:
    collection = Collections.get_collection(collection_id=999999)
except DaplaClientError as e:
    print(e)  # "Not found. Check that the ID is correct and the resource exists."
```

| HTTP Status | Message |
|---|---|
| 400 | Problem with supplied data (includes field-level validation details) |
| 401 | Authentication problem — token may have expired |
| 403 | Forbidden — no access to the resource |
| 404 | Not found — check the ID |
| 405 | Operation not allowed for current resource state |
| 409 | Conflict with existing data |

## API Reference

### Collections (read-only)

| Method | Description |
|---|---|
| `Collections.list_collections()` | List all published collections |
| `Collections.get_collection(collection_id)` | Get a single collection by ID |
| `Collections.search_by_name(name)` | Search by name (partial match) |
| `Collections.search_by_altinn_id(altinn_questionnaire_id)` | Search by Altinn questionnaire ID |
| `Collections.get_versions(collection_id)` | Get all versions of a collection |

### Drafts (CRUD)

| Method | Description |
|---|---|
| `Drafts.list_mine()` | List the current user's drafts |
| `Drafts.list_team_drafts()` | List drafts from all teams the user belongs to |
| `Drafts.get(draft_id)` | Get a draft by ID |
| `Drafts.create(draft)` | Create a new draft |
| `Drafts.update(draft_id, draft)` | Update an existing draft |
| `Drafts.delete(draft_id)` | Delete a draft |
| `Drafts.publish(draft_id)` | Publish a draft as a collection |

## Contributing

Contributions are very welcome.
To learn more, see the [Contributor Guide].

## License

Distributed under the terms of the [MIT license][license],
_Dapla Toolbelt Datafangst_ is free and open source software.

## Issues

If you encounter any problems,
please [file an issue] along with a detailed description.

## Credits

This project was generated from [Statistics Norway]'s [SSB PyPI Template].

[dapla-auth-client]: https://github.com/statisticsnorway/dapla-auth-client
[statistics norway]: https://www.ssb.no/en
[pypi]: https://pypi.org/
[ssb pypi template]: https://github.com/statisticsnorway/ssb-pypitemplate
[file an issue]: https://github.com/statisticsnorway/dapla-toolbelt-datafangst/issues
[pip]: https://pip.pypa.io/

<!-- github-only -->

[license]: https://github.com/statisticsnorway/dapla-toolbelt-datafangst/blob/main/LICENSE
[contributor guide]: https://github.com/statisticsnorway/dapla-toolbelt-datafangst/blob/main/CONTRIBUTING.md
[reference guide]: https://statisticsnorway.github.io/dapla-toolbelt-datafangst/reference.html

