Metadata-Version: 2.4
Name: dbt-prune
Version: 0.1.3
Summary: Find orphaned dbt models and snapshots across multiple manifests and open Copilot-powered cleanup tasks.
Author: Tarun Pass
Requires-Python: >=3.9
Requires-Dist: click<9,>=8.1
Requires-Dist: pydantic<3,>=2.8
Requires-Dist: pyyaml<7,>=6.0
Requires-Dist: requests<3,>=2.31
Provides-Extra: datahub
Requires-Dist: acryl-datahub>=0.14.1; extra == 'datahub'
Provides-Extra: dev
Requires-Dist: mypy<2,>=1.11; extra == 'dev'
Requires-Dist: pytest<9,>=8.3; extra == 'dev'
Requires-Dist: ruff<1,>=0.6; extra == 'dev'
Requires-Dist: types-pyyaml<7,>=6.0.12.20250516; extra == 'dev'
Requires-Dist: types-requests<3,>=2.32.4.20250611; extra == 'dev'
Provides-Extra: gcs
Requires-Dist: google-cloud-storage>=2.0.0; extra == 'gcs'
Description-Content-Type: text/markdown

# dbt-prune

`dbt-prune` is a pip-installable dbt-core companion CLI for finding dbt models and snapshots that are no longer referenced by anything in your current project, configured sibling projects, or dbt exposures.

## Installation

```bash
pip install dbt-prune
```

Optional integrations stay lightweight by default:

```bash
pip install 'dbt-prune[gcs]'
pip install 'dbt-prune[datahub]'
```

## Configuration

By default `dbt-prune` looks for `dbt_prune.config.yml` in the dbt project root passed to `--project-dir`.

```yaml
manifests:
  - name: other_project
    type: local
    config:
      path: ../other_project/target/manifest.json

datahub:
  enabled: false
  gms_server_env_var: DATAHUB_GMS_SERVER
  token_env_var: DATAHUB_TOKEN
  platform: bigquery
  env: PROD
```

### GCS manifest

Install the GCS extra and declare a `gcs` manifest entry:

```bash
pip install 'dbt-prune[gcs]'
```

```yaml
manifests:
  - name: analytics
    type: gcs
    config:
      project_id: data-mdm-prod
      bucket_name: tron-dbt-artifacts-prod
      object_name: orchestration/analytics/manifest.json
    excluded_packages:
      - customers
      - customer_platform
```

Optional GCS config fields:

| Field | Description |
|---|---|
| `credentials` | Path to a service-account JSON key file (uses ADC when omitted) |
| `impersonate_service_account` | Service account email to impersonate |

### Package filtering

Each manifest entry supports `excluded_packages` and `included_packages` (mutually exclusive). Nodes whose `package_name` is in `excluded_packages` are dropped from the manifest before orphan evaluation; `included_packages` keeps only matching nodes.

### Optional manifests

Set `optional: true` on a manifest entry to allow it to fail without aborting the run:

```yaml
manifests:
  - name: staging
    type: gcs
    optional: true
    config:
      project_id: my-project
      bucket_name: my-bucket
      object_name: staging/manifest.json
```

The config file stores the **names** of the environment variables only. The actual DataHub GMS server URL and token must come from the environment at runtime:

```bash
export DATAHUB_GMS_SERVER=https://datahub.example.com
export DATAHUB_TOKEN=...
```

## How orphan detection works

`dbt-prune` only parses `manifest.json` files. It does **not** run `dbt`, compile models, or introspect a live warehouse.

It loads:
- the current project's `target/manifest.json`
- any configured external manifests from local paths or `gs://` URIs
- exposure dependencies from every loaded manifest

A current-project model or snapshot is considered orphaned only when it has zero incoming references from any loaded manifest and is not referenced by any exposure. If DataHub checking is enabled, downstream lineage consumers also keep a node off the orphan list.

## Usage

### List all orphaned assets

```bash
dbt-prune ls --project-dir .
```

### Inspect one specific model

```bash
dbt-prune ls --project-dir . -s my_model
```

When `-s/--select` is used, `dbt-prune` reports whether the named model or snapshot is orphaned and explains why.

### Filter by package

Scope orphan detection to a specific dbt package within your project using `-p`/`--package`:

```bash
dbt-prune ls -p marketing_core
```

Combine with `-s`/`--select` to inspect a specific model within a package:

```bash
dbt-prune ls -p marketing_core -s some_model_name
```

The same flag works for `run`:

```bash
dbt-prune run -p marketing_core --dry-run
```

### JSON output

```bash
dbt-prune ls --format json
```

### Open Copilot cleanup tasks for every orphan

```bash
dbt-prune run --project-dir .
```

Behavior:
- scans all current-project models and snapshots by default
- opens one GitHub Copilot coding-agent task per orphan
- auto-detects the target `owner/repo` from `git remote get-url origin`
- requires `GITHUB_TOKEN` for GitHub API authentication

### Dry-run the cleanup plan

```bash
dbt-prune run --dry-run
```

### Restrict cleanup to one model

```bash
dbt-prune run -s my_model
```

### Limit the number of cleanup tasks

```bash
dbt-prune run --limit 5
```

## DataHub integration

When `datahub.enabled: true`, `dbt-prune` queries DataHub for downstream lineage before marking an otherwise unreferenced asset as orphaned.

Dataset URNs are built from each manifest node's materialized location:
`database.schema.alias` (falling back to `name` when alias is omitted). Ensure the manifest's `database` / `schema` / `alias` values match how your warehouse assets are ingested into DataHub.

Use `datahub.platform` and `datahub.env` to match your DataHub dataset URN convention:

```yaml
datahub:
  enabled: true
  gms_server_env_var: DATAHUB_GMS_SERVER
  token_env_var: DATAHUB_TOKEN
  platform: bigquery
  env: PROD
```

If the required environment variables are missing or DataHub is unreachable, `dbt-prune` warns and continues unless you pass `--strict`.

## GitHub Copilot coding-agent PR automation

`dbt-prune run` generates a clear problem statement for each orphaned asset and sends it to GitHub through `dbt_prune.pr_agent`. The default implementation creates a GitHub issue payload intended for Copilot coding-agent workflows, and the module is designed to be replaceable if your organization uses a dedicated Copilot task/session endpoint.

## Development

```bash
python -m pip install -e '.[dev]'
ruff check .
pytest
mypy dbt_prune
```
