Metadata-Version: 2.4
Name: demyst
Version: 0.1.0a2
Summary: Scientific integrity linter for academic code
Home-page: https://github.com/Hmbown/demyst
Author: Demyst Team
Author-email: Hunter Brown <hunter@demyst.dev>
License: MIT
Project-URL: Homepage, https://github.com/Hmbown/demyst
Project-URL: Documentation, https://github.com/Hmbown/demyst#readme
Project-URL: Repository, https://github.com/Hmbown/demyst
Project-URL: Issues, https://github.com/Hmbown/demyst/issues
Keywords: linting,scientific-computing,machine-learning,data-science,integrity,reproducibility,p-hacking,data-leakage,mirage,computational-mirages
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.20.0
Requires-Dist: PyYAML>=6.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: libcst>=1.0.0
Requires-Dist: rich>=13.0.0
Requires-Dist: langchain-core>=0.1.0
Requires-Dist: mcp>=1.0.0; python_version >= "3.10"
Requires-Dist: ipython>=8.0.0
Requires-Dist: scipy>=1.7.0
Provides-Extra: torch
Requires-Dist: torch>=1.9.0; extra == "torch"
Provides-Extra: jax
Requires-Dist: jax>=0.3.0; extra == "jax"
Requires-Dist: jaxlib>=0.3.0; extra == "jax"
Provides-Extra: tracking
Requires-Dist: wandb>=0.12.0; extra == "tracking"
Requires-Dist: mlflow>=1.20.0; extra == "tracking"
Provides-Extra: all
Requires-Dist: torch>=1.9.0; extra == "all"
Requires-Dist: jax>=0.3.0; extra == "all"
Requires-Dist: jaxlib>=0.3.0; extra == "all"
Requires-Dist: wandb>=0.12.0; extra == "all"
Requires-Dist: mlflow>=1.20.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=6.0.0; extra == "dev"
Requires-Dist: pytest-cov>=2.0.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: isort>=5.0.0; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"
Requires-Dist: hypothesis>=6.0.0; extra == "dev"
Requires-Dist: mutmut>=2.0.0; extra == "dev"
Requires-Dist: scipy>=1.7.0; extra == "dev"
Requires-Dist: pandas>=1.3.0; extra == "dev"
Requires-Dist: scikit-learn>=1.0.0; extra == "dev"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# Demyst

> **de·mys·ti·fy** /dēˈmistəˌfī/ — to make less obscure or confusing

[![PyPI version](https://badge.fury.io/py/demyst.svg)](https://badge.fury.io/py/demyst)
[![Tests](https://github.com/Hmbown/demyst/actions/workflows/ci.yml/badge.svg)](https://github.com/Hmbown/demyst/actions/workflows/ci.yml)
[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A scientific linter for research code. Like `black` for formatting and `mypy` for types, `demyst` checks **scientific logic**.

```bash
pip install demyst
demyst analyze ./src
```

## Status

| | |
|---|---|
| **Stability** | Alpha — actively developed, API may change |
| **Python** | 3.8, 3.9, 3.10, 3.11, 3.12 |
| **Ecosystems** | NumPy, pandas, scikit-learn, PyTorch, JAX, SciPy |
| **Philosophy** | Prefer false positives over silent failures — use `# demyst: ignore` to suppress |

## What It Catches

| Check | What It Detects | Example |
|-------|-----------------|---------|
| `leakage` | Train/test contamination | `fit_transform()` before `train_test_split()` |
| `mirage` | Variance-destroying reductions | `np.mean()` hiding outliers in your data |
| `hypothesis` | P-hacking, multiple comparisons | 20 t-tests without Bonferroni correction |
| `tensor` | Gradient death, normalization issues | Deep sigmoid chains, disabled BatchNorm stats |
| `units` | Dimensional mismatches | Adding meters to seconds |

## Try It in 30 Seconds

```bash
git clone https://github.com/Hmbown/demyst.git
cd demyst
pip install -e .
demyst leakage examples/ml_data_leakage.py
```

You'll see demyst catch the classic ML mistake: preprocessing data before splitting it.

## Sample Output

```text
$ demyst leakage examples/ml_data_leakage.py

──────────────────────────── Data Leakage Detected ─────────────────────────────

CRITICAL Line 47 in examples/ml_data_leakage.py
  fit_transform() called BEFORE train_test_split.
  Preprocessing learns from test data — your benchmark is invalid.

  45   X, y = load_medical_data()
  46   scaler = StandardScaler()
❱ 47   X_scaled = scaler.fit_transform(X)  # LEAKS TEST INFO
  48   X_train, X_test, y_train, y_test = train_test_split(X_scaled, y)

  Fix: Split first, then fit on train only:
       X_train, X_test = train_test_split(X)
       X_train = scaler.fit_transform(X_train)
       X_test = scaler.transform(X_test)

Summary: 1 critical issue
```

## Quick Examples

**Leakage** — the #1 ML benchmarking error:

```python
# WRONG: Leaks test statistics into training
scaler.fit_transform(X)
X_train, X_test = train_test_split(X_scaled)

# CORRECT
X_train, X_test = train_test_split(X)
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
```

**P-Hacking** — uncorrected multiple comparisons:

```python
# 20 tests at α=0.05 expects 1 false positive
for condition in conditions:
    if ttest(a[condition], b[condition]).pvalue < 0.05:
        print(f"{condition} significant!")  # No correction applied
```

## Usage

```bash
# Full analysis
demyst analyze your_code.py

# Individual guards
demyst mirage model.py
demyst leakage train.py
demyst hypothesis stats.py
demyst units physics.py
demyst tensor network.py

# Auto-fix mirages
demyst mirage model.py --fix

# CI mode
demyst ci . --strict
```

## Why Mirages Matter

These documented cases show how `np.mean()` hides critical information:

| Phenomenon | What Happened |
|------------|---------------|
| **Anscombe's Quartet** (1973) | Four datasets with identical mean (7.5) but completely different distributions |
| **Simpson's Paradox** (Berkeley 1973) | 44% male vs 35% female admission overall, but women admitted more in 4/6 departments |
| **Fat Tails in Finance** | Average daily return ~0.04% hides Black Monday's -22.6% single-day crash |
| **Outlier Masking** | Multiple outliers pull mean toward them, causing detection tests to fail |

Run `demyst mirage examples/real_world_mirages.py` to see detection in action.

## CI/CD

**GitHub Actions:**

```yaml
name: Demyst
on: [push, pull_request]
jobs:
  demyst:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install demyst
      - run: demyst ci . --strict
```

**Pre-commit:**

If you're already using `black` + `mypy` + `ruff`, drop this in next to them:

```yaml
repos:
  - repo: https://github.com/Hmbown/demyst
    rev: v0.1.0a1
    hooks:
      - id: demyst
```

See [`examples/configs/`](examples/configs/) for more templates.

## Suppressing False Positives

Use inline comments to suppress specific warnings:

```python
# Suppress all demyst warnings on this line
mean_value = np.mean(data)  # demyst: ignore

# Suppress only mirage warnings
dashboard_avg = np.mean(daily_views)  # demyst: ignore-mirage

# Suppress only leakage warnings  
scaler.fit_transform(X)  # demyst: ignore-leakage
```

Available suppressions: `ignore`, `ignore-mirage`, `ignore-leakage`, `ignore-hypothesis`, `ignore-tensor`, `ignore-unit`, `ignore-all`

## Configuration

Create `.demystrc.yaml`:

```yaml
profile: default  # Or: biology, physics, chemistry, economics

rules:
  mirage:
    enabled: true
    severity: critical
  leakage:
    enabled: true
    severity: critical

ignore_patterns:
  - "**/tests/**"
```

## Programmatic API

```python
from demyst import TensorGuard, LeakageHunter, HypothesisGuard, UnitGuard

source = open('model.py').read()
result = LeakageHunter().analyze(source)

if result['summary']['critical_count'] > 0:
    print("DATA LEAKAGE DETECTED")
```

## Design Principles

Silent failures in research code don't crash — they produce *wrong numbers that look right*. A model trains, metrics look good, paper gets submitted... then someone discovers the test set leaked into training. Demyst catches these before they become retractions.

| Principle | What It Means |
|-----------|---------------|
| **Yell early** | Prefers false positives over silent failures. Use `# demyst: ignore` to suppress. |
| **Static analysis** | AST-based heuristics + light dataflow. No runtime overhead, works on any Python. |
| **Actionable output** | Every warning includes the *why* and a concrete fix suggestion. |
| **Escape hatches** | Inline suppression (`# demyst: ignore-mirage`), config files, CI thresholds. |

**Detection capabilities:**

- **Mirage**: Detects 80+ NumPy array creators, tracks variable flow, checks for nearby variance operations
- **Leakage**: Tracks `fit`/`fit_transform` calls relative to `train_test_split`/`cross_val_score`
- **Hypothesis**: Counts statistical tests, checks for correction methods, detects p-value conditionals
- **Tensor**: Analyzes layer sequences for gradient death patterns, normalization misuse
- **Units**: Dimensional analysis via variable naming conventions and explicit annotations

## References

| Phenomenon | Finding | Source |
|------------|---------|--------|
| Anscombe's Quartet | Identical means hide different distributions | [Anscombe (1973)](https://en.wikipedia.org/wiki/Anscombe%27s_quartet) |
| Simpson's Paradox | Trends reverse when aggregated | [UC Berkeley (1975)](https://discovery.cs.illinois.edu/dataset/berkeley/) |
| Fat Tails | Normal assumptions hide crashes | [Mandelbrot (1963)](https://en.wikipedia.org/wiki/Fat-tailed_distribution) |
| Retraction Stats | 18.9% from computational errors | [PMC5395722](https://pmc.ncbi.nlm.nih.gov/articles/PMC5395722/) |

## Resources

- [Quick Start Guide](docs/quickstart.md)
- [Interactive Notebook](notebooks/quickstart.ipynb)
- [Configuration Templates](examples/configs/)
- [Full Documentation](docs/usage.md)

## License

MIT — See [LICENSE](LICENSE)

---

*"The first principle is that you must not fool yourself—and you are the easiest person to fool."* — Richard Feynman
