Metadata-Version: 2.4
Name: wraith-sas-lineage
Version: 0.2.0
Summary: SAS lineage extraction, schema mapping, and ODCS contract generation for the Ghost Stack.
Project-URL: Homepage, https://git.thomaspeoples.com/thomaspeoples/wraith-sas-lineage
Project-URL: Repository, https://git.thomaspeoples.com/thomaspeoples/wraith-sas-lineage.git
Project-URL: Documentation, https://www.thomaspeoples.com/gitea-repos/wraith-sas-lineage/
Project-URL: Issues, https://git.thomaspeoples.com/thomaspeoples/wraith-sas-lineage/issues
Project-URL: Changelog, https://git.thomaspeoples.com/thomaspeoples/wraith-sas-lineage/src/branch/main/CHANGELOG.md
Author-email: Thomas Peoples <hello@thomaspeoples.com>
License: MIT License
        
        Copyright (c) 2026 Thomas Peoples
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: analytics-engineering,data-contracts,ghost-stack,lineage,odcs,sas
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Code Generators
Requires-Python: >=3.12
Requires-Dist: click>=8.1
Requires-Dist: flask>=3.0
Requires-Dist: openpyxl>=3.1
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: commitizen; extra == 'dev'
Requires-Dist: detect-secrets; extra == 'dev'
Requires-Dist: genbadge[coverage]>=1.1.1; extra == 'dev'
Requires-Dist: mkdocs; extra == 'dev'
Requires-Dist: mkdocs-material; extra == 'dev'
Requires-Dist: mkdocstrings[python]; extra == 'dev'
Requires-Dist: poethepoet; extra == 'dev'
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: pydoc-markdown; extra == 'dev'
Requires-Dist: pymdown-extensions; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: ty; extra == 'dev'
Requires-Dist: typer; extra == 'dev'
Description-Content-Type: text/markdown

[![Documentation](https://img.shields.io/badge/docs-live-brightgreen)](https://www.thomaspeoples.com/gitea-repos/wraith-sas-lineage/)
![License](https://img.shields.io/badge/license-MIT-blue)
![Python](https://img.shields.io/badge/python-3.12%2B-blue)

# 👻 wraith-sas-lineage
### *SAS Lineage Extraction for the Ghost Stack*

**wraith-sas-lineage** extracts data lineage from SAS DI Studio code and verbose execution logs, maps it to Open Data Contract Standard (ODCS) contracts, and serves a local read-only lineage explorer. One JSON artefact connects all three tools.

Designed for large SAS estates running on Unix. Output feeds [wraith-modelgen](https://git.thomaspeoples.com/thomaspeoples/wraith-modelgen) to auto-generate dbt on BigQuery.

---

## Three tools. One seam.

```
.sas + .log files
      │
      ▼
  sas-parse  ──►  provenance.json
                       │
                       ▼
                   sas-map  ──►  lineage.db  +  ODCS contracts  +  gaps report
                                     │
                                     ▼
                                  sas-ui  ──►  localhost:5000
```

| Command | Input | Output |
|---|---|---|
| `sas-parse` | `.sas` files + verbose logs | `provenance.json` |
| `sas-map` | `provenance.json` | `lineage.db`, ODCS YAML contracts, `mapping_gaps.json` |
| `sas-ui` | `lineage.db` | Read-only lineage explorer |

---

## Install

```bash
uv tool install wraith-sas-lineage
```

Or for development:

```bash
git clone https://git.thomaspeoples.com/thomaspeoples/wraith-sas-lineage.git
cd wraith-sas-lineage
uv run poe setup
```

---

## Quick start

```bash
# Parse SAS code + logs
sas-parse \
  --code-dir /path/to/sas/code \
  --log-dir  /path/to/verbose/logs \
  --estate   acme-risk \
  --output   provenance.json

# Map to DB + ODCS contracts
sas-map \
  --provenance provenance.json \
  --db         lineage.db \
  --contracts-dir contracts/ \
  --gaps       mapping_gaps.json

# Browse lineage
sas-ui --db lineage.db
# → http://localhost:5000/flags      all extraction flags
# → http://localhost:5000/provenance upstream chain for a column
# → http://localhost:5000/lineage    downstream to all MARTs
```

---

## Design principles

**Flag-never-drop.** Unresolved macros, remote execution blocks, and hand-written code surface as `Flag` objects with reason, context, and snippet. Nothing is silently discarded.

**Log-first macro resolution.** `SYMBOLGEN`/`MLOGIC` log lines resolve `&macro_var` references before falling back to the code literal. Accurate lineage for DI Studio estates requires verbose logs.

**Single seam.** `provenance.json` is the only interface between tools. Each tool is independently runnable and testable.

**Content-hashed stable IDs.** Table, column, job, and lineage chain IDs are `SHA-256[:12]` of their canonical string. Same estate, same run, same IDs.

---

## Layer vocabulary

| SAS layer | Schema value |
|---|---|
| RAW | `RAW` |
| ODS (ODP) | `ODP` |
| STG (CDP) | `CDP` |
| MART (FDP) | `FDP` |

---

## Developer quality gate

```bash
uv run poe test       # pytest (interactive)
uv run poe test-ci    # pytest + coverage ≥ 80%
uv run poe lint       # ruff check
uv run poe format     # ruff format
uv run poe docs-serve # mkdocs at localhost:8000
```

All commits go through commitizen:

```bash
uv run cz commit
```

---

*Part of the [Ghost Stack](https://git.thomaspeoples.com/thomaspeoples). Sovereign. Self-hosted. No nonsense.*
