Metadata-Version: 2.5
Name: ad-data-pf
Version: 0.1.0
Summary: Tables and figures for a job-ad amenities project, run on Statistics Denmark microdata
Author-email: AskerNC <hms467@econ.ku.dk>
License-Expression: MIT
License-File: LICENSE
Requires-Python: >=3.12
Requires-Dist: loguru>=0.7
Requires-Dist: matplotlib>=3.10
Requires-Dist: numpy>=2.3
Requires-Dist: polars>=1.43
Description-Content-Type: text/markdown

# ad-data-pf

Tables (and later figures) for a research project on job-ad amenities, run on
Statistics Denmark (DST) microdata. The functions are written and tested
locally against simulated data, then `pip install`ed on the DST research
server, where the data are cleaned and the functions called.

All numbers in this package come from simulated data.

## Use

```python
from ad_data_pf import validation as val, save_tex, save_fig, setup_log

setup_log(log_folder, 'descriptives')      # logs the package version first
frames = {'aku': aku, 'jobads': jobads, 'jobads_linked': jobads_linked, 'aku_linked': aku_linked}
save_tex(val.desc_table(frames), out / 'tab_base_rates.tex')
save_tex(val.omission_table(aku_linked, ci='wilson'), out / 'tab_validation_measures.tex')
save_fig(val.prevalence_figure(aku_linked, aku), out / 'fig_sensitivity_occ_prevalence.pdf')
```

Tables return a bare `tabular`; figures return a matplotlib `Figure`.
Defaults (variables, labels, panels) sit at the top of each module and in
`ad_data_pf.labels`, and can be overridden per call. See the docstrings, and
`examples/` for every exhibit and for custom tables.

`validation` makes these exhibits:

| Exhibit | Function |
| --- | --- |
| Samples and base rates | `desc_table` (any number of frames and panels) |
| Confusion table | `confusion_table` (shares of N by default, or counts) |
| Validation measures by type | `omission_table` (Wilson or cluster-bootstrap intervals) |
| Measure by occupational prevalence | `prevalence_figure` (`measure='sens'` or `'fom'`) |
| Measures by group, robustness | `measures_table` (filters, frames or `variant(t=..., m=...)` per row) |
| Measures by linkage criterion | `ladder_figure` |

Inputs are checked: T and M must be Boolean without nulls, a row whose
columns exist in no frame or whose filter selects nothing raises, and cells
resting on fewer than 5 observations are left empty (DST disclosure).

On the server, install an exact version (`pip install ad-data-pf==0.1.0`) so a
rerun reproduces the same table.

## Layout

```text
src/ad_data_pf/
├── functions/          one module per topic: validation.py, ...
│                       imported from the top: from ad_data_pf import validation
├── simulation/         fake data per module (validation.py) and the files in data/
└── examples/
    └── validation/     code/ runs every table on the fake data;
                        logs/ and output/ hold what it writes
```

`simulation/` and `examples/` are installed with the package. To find them:

```bash
python -c "import ad_data_pf, pathlib; print(pathlib.Path(ad_data_pf.__file__).parent)"
python <that folder>/examples/validation/code/run_validation.py <output folder>
```

Without an output folder the example writes next to itself, which fails if
the installed package is read-only.

## Adding a module

1. `functions/<name>.py`: importable at once as `ad_data_pf.<name>`
2. `simulation/<name>.py`: `simulate()`, `save()`, `load()`
3. `examples/<name>/code/run_<name>.py`, plus `logs/.gitkeep`
4. `tests/test_<name>.py`

## Development

The package lives in `ad_data_pf/` of the (private) project repository; run
everything below from that folder.

```bash
uv sync                                          # environment in .venv
uv run pytest                                    # also compiles the LaTeX if latexmk is found
uv run python src/ad_data_pf/simulation/validation.py   # redraw the shipped fake data
```

The lower bounds in `pyproject.toml` are the versions on the DST server. To
test against exactly those, use a separate environment so `uv.lock` is not
rewritten:

```bash
cp uv.lock "$TEMP/uv.lock"
UV_PROJECT_ENVIRONMENT="$TEMP/venv-server" uv sync --resolution lowest-direct
UV_PROJECT_ENVIRONMENT="$TEMP/venv-server" uv run --no-sync pytest
cp "$TEMP/uv.lock" uv.lock
```

GitHub Actions does both on every push that touches `ad_data_pf/`
(`.github/workflows/ad_data_pf_test.yml` at the repository root).

## Release

Bump `version` in `pyproject.toml`, commit, then push a tag
`ad_data_pf-v<version>`; GitHub Actions checks that the tag matches the
version, runs the tests, builds and publishes to PyPI (trusted publishing,
`.github/workflows/ad_data_pf_publish.yml`):

```bash
git tag ad_data_pf-v0.1.1 && git push origin ad_data_pf-v0.1.1
```
