Metadata-Version: 2.4
Name: dbt-tester
Version: 0.2.3
Summary: Static analysis and health report generator for dbt projects
Author-email: Vinoth J <career.vinothj@gmail.com>
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Requires-Dist: jinja2>=3.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Dynamic: license-file

# dbt-tester

A powerful static analysis tool for dbt projects that scans models, schemas, and sources to find issues like missing tests, undocumented columns, hardcoded references, and more. Features an interactive HTML report with D3.js lineage visualization.

## What is dbt-tester?

DBT TESTER helps you:
- Find models without tests
- Identify undocumented columns
- Catch hardcoded table references
- Verify data quality rules
- **Visualize model lineage** with interactive D3.js graph
- **Track test execution** results and coverage
- Generate beautiful HTML/JSON/Console reports

## Quick Start (5 minutes)

### Step 1: Install

```bash
# From PyPI (recommended)
pip install dbt-tester

# Or from source
git clone <repo-url>
cd dbt-tester
python -m venv .venv
source .venv/bin/activate
pip install -e .
```

### Step 2: Run Your First Scan

```bash
# Scan a dbt project
dbt-tester run /path/to/your/dbt/project

# Or scan current folder
dbt-tester run .
```

### Step 3: Understand Results

```
DBT TESTER v0.2.3
Version: 0.2.3 | Created by: VINOTH J

SYSTEM STATUS: X CRITICAL
Models: 24 | Sources: 0 | Exposures: 0

ISSUE MATRIX:
  ERROR: 82
  WARN:  41

LINEAGE GRAPH:
  t1_drs_status_master    model  incremental  root
  t1_center_master        model  incremental  root
  ...

TEST RESULTS:
  PASSED: 0 | FAILED: 0 | PASS RATE: 0%

TEST COVERAGE:
  Models with Tests: 0 | Total Models: 24 | Coverage: 0%
```

## Features Overview

### 1. Interactive Lineage Graph (HTML)
- **D3.js powered** force-directed graph
- **Zoom in/out** with buttons or mouse wheel
- **Pan/drag** to move around
- **Drag nodes** to rearrange
- **Click nodes** for detailed modal
- **Export SVG** to save graph
- Color-coded by resource type (model/source/exposure)

### 2. Test Results Dashboard
- Pass/Fail/Skip counts from `run_results.json`
- Pass rate with visual progress bar
- Tests grouped by model
- Click "Info" for explanation

### 3. Test Coverage Tracking
- % of models with tests defined
- Materialization breakdown
- Click for detailed stats
- Low coverage warnings with suggestions

### 4. Multiple Output Formats
| Format | Command | Best for |
|--------|---------|----------|
| Terminal | `--format console` | Quick review |
| HTML | `--format html --output report.html` | Share with team |
| JSON | `--format json --output report.json` | CI/CD automation |

### 5. Exit Control for CI/CD

| Option | When it exits with code 1 |
|--------|---------------------------|
| `--fail-on error` | If any ERROR found |
| `--fail-on warning` | If any WARNING or ERROR found |

## Command Options

### Basic Options

| Command | What it does |
|---------|--------------|
| `dbt-tester run .` | Scan current directory |
| `dbt-tester run /path/to/project` | Scan specific folder |
| `dbt-tester run . -v` | Show fix instructions |
| `dbt-tester --version` | Show version |

### Generate Reports

```bash
# Console output
dbt-tester run . --format console

# HTML with interactive graph
dbt-tester run . --format html --output report.html

# JSON for automation
dbt-tester run . --format json --output report.json
```

### CI/CD Integration

```bash
# Exit 1 if any error
dbt-tester run . --fail-on error

# Create report for artifacts
dbt-tester run . --format html --output dbt-report.html --fail-on warning
```

## Understanding Output

### Status Types

- **X CRITICAL** = Must fix (will break builds)
- **! WARNING** = Should fix (data quality risk)
- **OK HEALTHY** = No issues found

### Common Issues

| Issue | Meaning | How to Fix |
|-------|---------|------------|
| ERROR: Model has no tests | Add tests in schema.yml | Add `tests: - not_null` |
| ERROR: Missing description | Add doc in schema.yml | Add `description: "..."` |
| ERROR: Hardcoded table ref | Use ref() instead | Replace `db.table` with `{{ ref('model') }}` |
| WARN: Column missing not_null | Add not_null test | Add test for *_id columns |

## Getting Real Test Data

To see actual test results and coverage in reports:

```bash
cd your-dbt-project

# 1. Compile to generate manifest.json
dbt compile

# 2. Run models
dbt run

# 3. Run tests to get test results
dbt test

# 4. Now run dbt-tester with real data
dbt-tester run . --format html --output report.html
```

The report will now show:
- ✅ Test execution results (pass/fail)
- ✅ Actual test coverage %
- ✅ Lineage graph from manifest
- ✅ Model materialization stats

## Upgrading to a New Version

When a new version is released:

```bash
# Upgrade to the latest version
pip install --upgrade dbt-tester

# Or use
pip install -U dbt-tester
```

Check your current version:
```bash
pip show dbt-tester
dbt-tester --version
```

See available versions:
```bash
pip index versions dbt-tester
```

## For Developers

```bash
# Run tests
pytest

# Test with sample project
dbt-tester run tests/fixtures/basic_project --format console

# Build package
python -m build
```

## Publishing to PyPI

To publish a new version:

```bash
# 1. Update version in:
#    - pyproject.toml (version = "0.2.3")
#    - dbt_tester/__init__.py (__version__ = "0.2.3")

# 2. Clean old builds
rm -rf build dist *.egg-info

# 3. Build package
python -m build

# 4. Upload to PyPI (IMPORTANT: use version pattern)
twine upload dist/dbt_tester-*

# Or upload specific version:
twine upload dist/dbt_tester-0.2.3-py3-none-any.whl dist/dbt_tester-0.2.3.tar.gz
```

Set credentials first:
```bash
export TWINE_USERNAME=__token__
export TWINE_PASSWORD=pypi-xxxxxxxxxxxxxxxxxxxxxxxxxxxx
```

## Troubleshooting

### "No matching distribution found"
Your Python is too old. Use Python 3.9+:
```bash
python3 -m venv .venv
source .venv/bin/activate
pip install dbt-tester
```

### "externally-managed-environment"
Use virtual environment:
```bash
python -m venv .venv
source .venv/bin/activate
pip install -e .
```

### Test Results show 0
Run `dbt test` in your dbt project first to generate test results.

### Coverage shows 0%
Run `dbt compile` to generate manifest.json with test data.

## License

MIT License - Created by VINOTH J

---

*Static analysis for dbt projects - Made with ❤️*
