Metadata-Version: 2.4
Name: gcp-pipeline-ref-cdp
Version: 1.0.29
Summary: GCP Pipeline Reference: Generic CDP Transformation — dbt models for FDP to CDP (JOIN pattern across all 3 FDP tables)
License: MIT
Project-URL: Homepage, https://github.com/enrichmeai/gcp-pipeline-reference
Project-URL: Documentation, https://github.com/enrichmeai/gcp-pipeline-reference/tree/main/deployments/fdp-to-consumable-product
Project-URL: Repository, https://github.com/enrichmeai/gcp-pipeline-reference
Project-URL: Framework, https://pypi.org/project/gcp-pipeline-framework/
Keywords: gcp,bigquery,dbt,cdp,transformation,reference,example
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: dbt-bigquery>=1.5.0
Requires-Dist: gcp-pipeline-core==1.0.29
Requires-Dist: gcp-pipeline-transform==1.0.29
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"

# fdp-to-consumable-product

**Deployment type:** BigQuery (dbt)
**Layer:** CDP — Consumable Data Product
**Pattern:** JOIN across 3 FDP tables → 1 CDP table

## Overview

This deployment builds the **CDP (Consumable Data Product)** layer by joining all three FDP tables from `fdp_generic` into a single denormalised view per customer:

```
fdp_generic.event_transaction_excess    ─┐
fdp_generic.portfolio_account_excess    ─┼──► cdp_generic.customer_risk_profile
fdp_generic.portfolio_account_facility  ─┘
```

The `customer_risk_profile` CDP table is the primary source for the `mainframe-segment-transform` pipeline, which reads each row and writes an outbound fixed-width segment file back to GCS for downstream mainframe consumption.

## Data Flow

```
ODP (odp_generic) ──► FDP (fdp_generic) ──► CDP (cdp_generic) ──► GCS segment files
       ↑                    ↑                      ↑                      ↑
  Dataflow/Beam          dbt JOIN/MAP           dbt JOIN          Dataflow/Beam
  (ingestion)          (bigquery-to-         (this deployment)   (mainframe-segment-
                       mapped-product)                            transform)
```

## CDP Table: `cdp_generic.customer_risk_profile`

One row per customer per extract date. Combines:

| Source FDP Table | Fields |
|-----------------|--------|
| `event_transaction_excess` | customer identity, account exposure, current balance |
| `portfolio_account_excess` | risk decision, score, decision outcome |
| `portfolio_account_facility` | loan amount, interest rate, term, facility status |

Derived field `cdp_segment`:
- `ACTIVE_APPROVED` — decision approved + positive balance
- `DECLINED` — decision declined
- `REFERRED` — decision referred for manual review
- `PENDING` — no decision yet

## Structure

```
deployments/fdp-to-consumable-product/
├── pyproject.toml
├── Dockerfile                   # generic-cdp-transformation image
├── cloudbuild.yaml
├── dbt/
│   ├── dbt_project.yml
│   ├── packages.yml
│   └── models/
│       ├── staging/fdp/         # Thin wrappers over fdp_generic source tables
│       │   ├── _fdp_sources.yml
│       │   ├── stg_fdp_event_transaction_excess.sql
│       │   ├── stg_fdp_portfolio_account_excess.sql
│       │   └── stg_fdp_portfolio_account_facility.sql
│       └── cdp/                 # CDP output table
│           ├── _generic_cdp_models.yml
│           └── customer_risk_profile.sql
├── dbt/
│   └── macros/
│       └── cdp_quality_checks.sql
└── tests/
```

## Running Locally

```bash
# Setup venv
./scripts/setup_deployment_venv.sh fdp-to-consumable-product

# Activate and install dbt deps
source deployments/fdp-to-consumable-product/venv/bin/activate
cd deployments/fdp-to-consumable-product/dbt
dbt deps

# Run CDP models
export GCP_PROJECT_ID=joseph-antony-aruja
dbt run --target dev

# Run tests
dbt test
```

## Docker Build

```bash
cd /path/to/gcp-pipeline-reference
gcloud builds submit \
  --config deployments/fdp-to-consumable-product/cloudbuild.yaml \
  --substitutions _LIBRARY_VERSION=1.0.29 \
  .
```
