Metadata-Version: 2.4
Name: dansk-register-platform
Version: 0.18.2
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
Classifier: Typing :: Typed
Requires-Dist: polars>=1.41,<1.45
Requires-Dist: pydantic>=2
Requires-Dist: rich>=13
Requires-Dist: rich-argparse>=1.8
Requires-Dist: sas7bdat-polars==0.9.2
Requires-Dist: matplotlib>=3.8
Requires-Dist: dansk-register-platform[study1] ; extra == 'all'
Requires-Dist: dansk-register-platform[study2] ; extra == 'all'
Requires-Dist: dansk-register-platform[study3] ; extra == 'all'
Requires-Dist: pyarrow>=13 ; extra == 'repair'
Provides-Extra: all
Provides-Extra: repair
Provides-Extra: study1
Provides-Extra: study2
Provides-Extra: study3
License-File: LICENSE
Summary: Danish register analysis platform (dansk_register) and the study pipelines built on it: parental income, societal costs, and educational and labour-market trajectories
Keywords: epidemiology,registers,denmark,survival-analysis,polars
Author-email: Tobias Kragholm <tkragholm@gmail.com>
License-Expression: MIT
Requires-Python: >=3.12
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Issues, https://github.com/tkragholm/register-studies/issues
Project-URL: Repository, https://github.com/tkragholm/register-studies

# dansk-register-platform

A platform for epidemiological studies on the Danish national registers, and
the study pipelines built on it.

The package contains no data. It is the code that reads a register delivery
inside a secure research environment, builds cohorts and outcomes from it, and
estimates and reports the results.

## What is in it

| Package | Role |
|---|---|
| `dansk_register` | The reusable platform: register catalog and I/O, the staged-pipeline runner, matching, estimation, disclosure control, figure styling |
| `parental_income` | Study I: parental income after severe chronic disease in childhood |
| `societal_costs` | Study II: societal costs of severe chronic disease in childhood |
| `edu_labour_trajectories` | Study III: educational and labour-market trajectories from age 18 |
| `register_studies` | The orchestration command line |

The dependency direction is one way and enforced by a test: the platform never
imports a study, and studies never import each other. Code that turns out to be
general is lifted into `dansk_register` rather than shared sideways.

The register runner and the estimators are Rust, compiled into the wheel as one
extension module. The methods crates it builds on, cohort matching,
semicontinuous models, staggered difference-in-differences and the chronic
disease phenotype, are published separately and consumed by version.

## Install

```
pip install dansk-register-platform                 # everything, all three studies
pip install "dansk-register-platform[study2]"       # names a study; installs the same files
pip install "dansk-register-platform[all]"          # all three, explicitly
```

Before 0.7.0 the distribution was published as `dansk-register`. The import
packages did not change; only the name on `pip install` did. The old name still
resolves, to a distribution that stopped being updated at the rename.

The estimators are written in polars and validated against R, so the pipeline
carries no numeric stack of its own. matplotlib is a base dependency because
numpy arrived on every real run anyway; what keeps the boundary real is that its
imports live under a `figures` package and a contract test fails the build if one
escapes into an estimator. Without a renderer every result is still produced and
the figure stage records that it could not draw.

`repair` is the only optional dependency: pyarrow, for `register-repair-utf8`
alone. Install it with `pip install 'dansk-register-platform[repair]'`. The same
kind of contract test keeps pyarrow out of everything else.

## Commands

```
edu-labour --list-profiles                          # what each profile runs
edu-labour-preflight --raw-root PATH                # what a delivery contains
edu-labour --output-dir RUN --profile production \
    --raw-root PATH --allow-expensive-stages --resume
edu-labour-figures --bundle RUN/outputs/export --out figures

societal-costs --capability PATH                     # what this study needs vs. what a delivery has
societal-costs --output-dir RUN --profile production \
    --raw-root PATH --allow-expensive-stages --diagnostics
societal-costs secondary_sensitivity --output-dir RUN   # the arm no profile runs

register-repair-utf8 PATH --report utf8_scan.csv    # find invalid UTF-8 in converted Parquet
register-repair-utf8 PATH --fix --report utf8_fix.csv
```

`--allow-expensive-stages` is required for `production`: `matching` scans the
whole delivery, and without the flag the runner refuses rather than silently
starting a multi-hour match. `--profile production` runs 19 stages but not
`secondary_sensitivity`, the analyse-only sensitivity and subgroup arm, which
reads a completed run and writes into a subdirectory of it.

`register-repair-utf8` needs the `repair` extra and says so if it is missing.

Add `--dry-run` to any `edu-labour` invocation to resolve the plan and settings
without touching data.

## Design notes

**Stages talk through files.** Every stage writes named artifacts into a run
directory and reads its inputs from there, so any stage can be re-run alone
against a previous run's outputs, and `--resume` can skip what is already done.

**Estimators are written against polars.** Aalen-Johansen, Fine-Gray, Cox and
the multi-state occupancy are implemented directly rather than taken from a
modelling library, and each is validated against reference values from R with
those values checked into the tests. That keeps the numbers checkable line by
line and avoids a dependency that may not install in an offline environment.

**Disclosure control is part of the pipeline.** Results leave the secure
environment as a bundle of CSV tables with small cells suppressed and the
estimates resting on them blanked. Figures are rendered from that bundle rather
than from the raw artifacts, so a figure cannot show what the table beside it
was not cleared to show.

## What this covers, and what it does not

Secure-environment research is usually described with the [Five
Safes](https://fivesafes.org/), and it is worth saying plainly which of the five
a piece of software can touch at all.

| | |
|---|---|
| **Safe projects** | Your approval. Nothing here. |
| **Safe people** | Your institution's authorisation. Nothing here. |
| **Safe settings** | The host's environment, Statistics Denmark or the Danish Health Data Authority. Nothing here. |
| **Safe data** | **Partly here.** Person identifiers are replaced with a study-local key, and an export is refused outright if a CPR-shaped value reaches it. |
| **Safe outputs** | **Partly here.** Small cells are suppressed, dependent estimates blanked, file types and sizes checked against the transfer rules, and every bundle carries a manifest saying what was done and what was not. |

"Partly" is the operative word in both rows. The output checks are cell-by-cell
and cannot see a small number recovered by differencing across tables; each
bundle's `MANIFEST.txt` says so in its own text. Running this framework is not a
substitute for your host's output review, and it does not make an
unapproved analysis approved.

## Status

Research code, developed alongside the studies it implements. The public API is
not stable.

## Licence

MIT. See `LICENSE`.

