Metadata-Version: 2.4
Name: dbt-4ps-generator
Version: 0.1.1
Summary: Generate dbt staging models for 4PS Construct (Business Central) from bc2adls CDM manifests
Project-URL: Repository, https://github.com/misja-pronk/dbt-4ps-staging
Project-URL: Documentation, https://misja-pronk.github.io/dbt-4ps-staging/
Project-URL: Changelog, https://github.com/misja-pronk/dbt-4ps-staging/blob/main/CHANGELOG.md
Author: Misja Pronk
License-Expression: MIT
License-File: LICENSE
Keywords: 4ps,bc2adls,business-central,cdm,databricks,dbt
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Database
Requires-Python: >=3.13
Requires-Dist: databricks-sdk>=0.50.0
Requires-Dist: humps>=0.2.2
Requires-Dist: pydantic>=2.11.3
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: rich>=13.0.0
Requires-Dist: shandy-sqlfmt[jinjafmt]>=0.26.0
Requires-Dist: sqlglot>=26.15.0
Requires-Dist: typer>=0.15.2
Description-Content-Type: text/markdown

# dbt-4ps-generator

[![PyPI](https://img.shields.io/pypi/v/dbt-4ps-generator.svg)](https://pypi.org/project/dbt-4ps-generator/)

CLI that generates dbt staging models from a [bc2adls](https://github.com/Bertverbeek4PS/bc2adls) CDM export. For every entity in the manifest it writes:

- `stg_4ps__<table>.sql` — a `SELECT ... FROM STREAM READ_FILES(...)` model with the column types from the CDM metadata and columns renamed to snake_case (`CreditLimitLCY-20` → `credit_limit_lcy`)
- `_4ps__models.yaml` — model + column descriptions, and a `dbt_utils.unique_combination_of_columns` test on each table's primary key

Full documentation: <https://misja-pronk.github.io/dbt-4ps-staging/>

## Install

```bash
uvx dbt-4ps-generator --help          # run once, ephemerally
uv tool install dbt-4ps-generator     # install the command on PATH
```

Or from a checkout: `uv run dbt-4ps-generator` in this directory.

## Usage

Download the CDM metadata from the Unity Catalog volume bc2adls exports to (authenticates via `DATABRICKS_HOST`/`DATABRICKS_TOKEN` or a `~/.databrickscfg` profile):

```bash
dbt-4ps-generator download \
  --volume-path /Volumes/<catalog>/<schema>/<volume> \
  --output-directory ../_cdm
```

Generate the models:

```bash
dbt-4ps-generator generate \
  --manifest ../_cdm/deltas.manifest.cdm.json \
  --output-directory ../dbt_4ps_staging_package/models/staging/4ps
```

Options (see `--help`): `--model-prefix` (default `stg_4ps__`), `--schema-prefix` (default `_4ps__`), and `--no-file-metadata-column` to omit the `_metadata` source-file column.

## Development

```bash
uv sync
uv run pytest
uv run ruff check . && uv run ruff format --check .
uv run ty check
```

## Layout

| File (`src/dbt_4ps_generator/`) | Purpose |
|---|---|
| `cli.py` | Typer CLI (`generate`, `download`) |
| `cdm.py` | Pydantic models for the CDM manifest and entity files |
| `converters.py` | CDM → dbt/SQL conversions (types, names, READ_FILES options) |
| `dbt.py` | Builders that write the .sql models and schema yaml |
| `sql.py` | SQL primitives (columns, options, streaming-table DDL builder) |
| `databricks_helper.py` | Volume download via the Databricks SDK |
