Metadata-Version: 2.4
Name: ancestryaudit
Version: 0.3.1
Summary: Bias detection and correction framework for genomic cancer AI
Home-page: https://github.com/DanYerga/ancestryaudit
Author: Dana Yergaliyeva
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: scikit-learn>=1.0
Requires-Dist: numpy>=1.21
Requires-Dist: pandas>=1.3
Requires-Dist: scipy>=1.7
Requires-Dist: shap>=0.41
Requires-Dist: matplotlib>=3.4
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# AncestryAudit

**Bias detection and correction framework for genomic cancer AI.**

Detects ancestry-linked performance gaps in copy number variation (CNV)-based
cancer classifiers, applies McNemar's paired test for correction validation,
and generates structured audit reports.

Developed from research on ancestry bias in TCGA-LIHC/STAD/ESCA classification
(Yergaliyeva, 2026).

> **v0.3.0 — Statistical overhaul:** Original bootstrap t-test replaced with
> label-permutation test (FPR: 85% → 0%). Correction module rewritten around
> McNemar's paired test. Power analysis added. Full methodology in paper.

---

## Installation

```bash
pip install ancestryaudit
# or from source:
pip install -e .
```

---

## Input Format

AncestryAudit works on any CNV feature matrix:

| Format | Shape | Notes |
|--------|-------|-------|
| `np.ndarray` | `(n_samples, n_genes)` | Continuous copy-number values |
| `pd.DataFrame` | `(n_samples, n_genes)` | Column names = gene identifiers |

**Column values:** continuous copy number (e.g. TCGA ABSOLUTE pipeline output,
where 2.0 = normal diploid, >2 = amplification, <2 = deletion).

**Labels:** binary integer (0 or 1), one per sample.

**Models:** any scikit-learn compatible estimator with `fit` / `predict` interface.

---

## Quick Start

```python
from ancestryaudit import AncestryAuditFramework
from sklearn.linear_model import LogisticRegression

framework = AncestryAuditFramework()

# Step 0: Check if you have enough data (run this first)
power = framework.power_analysis(n_source=451, n_target=242, expected_gap_pp=3.0)
# → UNDERPOWERED: need n_target≈836 for 80% power at 3pp

# Step 1: Detect ancestry-linked performance gap
report = framework.audit(
    LogisticRegression(max_iter=1000),
    X_western, y_western,
    X_asian,   y_asian
)
print(f"Gap: {report.gap_pp:.2f}pp, p={report.p_value:.4f}")
print(f"Recommendation: {report.recommendation}")
```

Output:
```
Gap: +2.39pp, p=0.0069
Recommendation: correction_required
```

---

## Full Pipeline

```python
from ancestryaudit import AncestryAuditFramework
from sklearn.linear_model import LogisticRegression

framework = AncestryAuditFramework(
    random_state=42,
    threshold_pp=2.0,
    threshold_p=0.05
)

# ── Step 0: Power analysis (before anything else) ──────────────────────────
power = framework.power_analysis(
    n_source=451, n_target=242, expected_gap_pp=3.0
)
# Tells you whether your data is sufficient before you run the audit

# ── Step 1: Filter population-stratification noise (optional) ──────────────
X_western_filtered, kept_genes, filter_log = framework.filter_stratification_noise(
    X_western_df, gene_list
)

# ── Step 2: Audit ──────────────────────────────────────────────────────────
audit_report = framework.audit(
    LogisticRegression(max_iter=1000),
    X_western_filtered, y_western,
    X_asian_filtered,   y_asian
)
print(audit_report)
# AuditReport(gap=+2.39pp, p=0.0069, d=1.52,
#             null_CI=[-2.10, 2.15], recommendation='correction_required')

# ── Step 3: Correct ────────────────────────────────────────────────────────
if audit_report.recommendation == "correction_required":
    corrected_model, correction_report = framework.correct(
        LogisticRegression(max_iter=1000),
        X_western_filtered, y_western,
        X_asian_labeled,    y_asian_labeled,
        n_samples=75
    )
    print(correction_report)
    # CorrectionReport(delta=+1.43pp, McNemar p=0.031,
    #                  direction='fine-tuned better')

# ── Step 4: Validate ───────────────────────────────────────────────────────
validation_report = framework.validate(
    corrected_model,
    X_asian_holdout, y_asian_holdout
)

# ── Step 5: Report ─────────────────────────────────────────────────────────
framework.generate_report("my_audit_report.json")
framework.summary()
```

---

## API Reference

### `AncestryAuditFramework`

| Method | Description | Returns |
|--------|-------------|---------|
| `power_analysis(n_source, n_target, expected_gap_pp)` | Check data sufficiency before audit | `dict` |
| `audit(model, X_source, y_source, X_target, y_target)` | Detect gap (permutation test) | `AuditReport` |
| `filter_stratification_noise(X, gene_list)` | Remove OR/pseudogene columns | `(X_filtered, kept_genes, filter_log)` |
| `correct(model, X_source, y_source, X_target_labeled, y_target_labeled, n_samples)` | Fine-tune + McNemar test | `(corrected_model, CorrectionReport)` |
| `validate(corrected_model, X_holdout, y_holdout)` | Post-correction audit | `ValidationReport` |
| `generate_report(save_path)` | Full JSON report | `dict` |
| `summary()` | Print pipeline summary | `str` |

### `AuditReport` fields

| Field | Type | Description |
|-------|------|-------------|
| `gap_pp` | float | Accuracy gap in percentage points (positive = source better) |
| `p_value` | float | Two-sided p-value from **label-permutation test** |
| `cohen_d` | float | Between-group effect size (pooled SD of per-sample correctness) |
| `null_ci` | tuple | 2.5/97.5 percentiles of permutation null — **not a CI on the gap** |
| `source_accuracy` | float | Model accuracy on held-out source data |
| `target_accuracy` | float | Model accuracy on target data |
| `n_source` | int | Source sample count |
| `n_target` | int | Target sample count |
| `recommendation` | str | `"correction_required"` or `"no_action"` |

### `CorrectionReport` fields

| Field | Type | Description |
|-------|------|-------------|
| `delta_pp` | float | Accuracy improvement on holdout (pp) |
| `p_value` | float | McNemar's test p-value |
| `n_used` | int | Target samples used for fine-tuning |
| `n_holdout` | int | Holdout samples used for evaluation |
| `mcnemar` | dict | b, c, p_value, direction, test_used |
| `refit_robustness` | dict | 5 refits varying model random_state (split fixed) |
| `direction_confirmed` | bool | True if McNemar p<0.05 and fine-tuned better |
| `baseline_accuracy` | float | Source-only accuracy on full target |
| `corrected_accuracy` | float | Fine-tuned accuracy on full target |

### `ValidationReport` fields

| Field | Type | Description |
|-------|------|-------------|
| `pre_gap` | float | Performance gap before correction (pp) |
| `post_gap` | float | Performance gap after correction (pp) |
| `correction_magnitude` | float | pre_gap - post_gap |
| `improvement_pp` | float | Accuracy improvement on target (pp) |

---

## Statistical Design

### audit() — Permutation test

**Why not bootstrap?** The original implementation used a bootstrap t-test
that produced 85% false positive rate on null data (identical distributions).
Root cause: testing whether a point estimate's resampling variance excludes
zero is circular — it will always reject H0 on finite data regardless of
whether a true effect exists.

**Correct approach:** Label-permutation test. Shuffle which samples are
"source test" vs "target" 1000 times, recompute the gap under each shuffle,
locate the observed gap in that null distribution. Verified: 0% FPR across
800 null trials.

### correct() — McNemar's paired test

**Why not bootstrap on folds?** Three attempts at fold-based bootstrap
(independent seeds, StratifiedKFold, RepeatedStratifiedKFold) all produced
inflated FPR (22–26%) due to holdout overlap between folds at n_target=242.

**Correct approach:** McNemar's exact/chi-square test on a single fixed
holdout. Tests whether the two classifiers disagree asymmetrically — exactly
the right question. Verified: 0% FPR across 500 null trials.

---

## Power Analysis

**Critical:** run `power_analysis()` before `audit()`. At TCGA-scale Asian
cohorts (n≈242), the minimum detectable effect is ~8–11pp. Realistic
ancestry-linked gaps are 1–5pp. The test is structurally underpowered
for this problem at current sample sizes.

```python
result = framework.power_analysis(
    n_source=451, n_target=242, expected_gap_pp=3.0
)
# → UNDERPOWERED
# → n_target needed: ≈836 (both sides must scale together)
# → Minimum viable study: n_target≥750, n_source≥1,500
```

Reference: Saldanha et al. (2024, Nature Medicine) documented 3–16pp
ancestry-linked gaps in TCGA-trained cancer AI. Our MDE of 8.47pp exceeds
the upper end of this range.

---

## Filtering Details

`filter_stratification_noise` removes three gene categories:

- **Olfactory receptor genes** (`OR*`) — ancestry-linked CNV unrelated to cancer
- **Pseudogenes** (`*P`, `*P1`, `*P2`, …) — non-functional, high stratification signal
- **Uncharacterized loci** (names containing `.`) — no biological interpretation

**Methods disclosure:** Feature space defined using all samples prior to
train/test split (bounded data snooping). No label information used.
(Kaufman et al., 2012)

---

## Expected Results

> **Disclaimer:** Results depend on model, train/test split, and preprocessing.
> Directional consistency (not numerical identity) with paper results is the
> correct validation criterion.

**Null calibration (verified):**
- `audit()` permutation test: 0% FPR across 800 null trials
- `correct()` McNemar test: 0% FPR across 500 null trials

**Paper reproduction:**

| Metric | Paper | Library |
|--------|-------|---------|
| Mean PGI (LIHC/STAD) | +2.39pp | +2.393pp |
| Algorithms positive | 7/7 | 7/7 |
| p-value (permutation) | 0.0048 | verified |

---

## Tests

```bash
python tests/test_null_calibration.py   # FPR ≤ 20% on null data
python tests/test_power_calibration.py  # flip_rate calibration regression
```

---

## Citation

```
Yergaliyeva, D. (2026). Statistical pitfalls in ancestry-stratified
cancer genomic AI: Power analysis, circular-inference correction,
and structural confounds in TCGA CNV data.
[Manuscript in preparation]
```

---

## License

MIT License. Copyright (c) 2026 Dana Yergaliyeva.
