Metadata-Version: 2.4
Name: prepx
Version: 1.0.0
Summary: One-line DataFrame cleaning and EDA for pandas with advanced outlier detection, imputation, and drift analysis.
Author: prepx Contributors
License: MIT
Project-URL: Homepage, https://github.com/anomalyco/prepx
Project-URL: Repository, https://github.com/anomalyco/prepx
Keywords: pandas,data-cleaning,eda,outliers,imputation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: pandas>=1.5
Requires-Dist: numpy>=1.21
Provides-Extra: advanced
Requires-Dist: scikit-learn>=1.3; extra == "advanced"
Requires-Dist: scipy>=1.11; extra == "advanced"
Provides-Extra: viz
Requires-Dist: matplotlib>=3.5; extra == "viz"
Requires-Dist: seaborn>=0.12; extra == "viz"
Provides-Extra: plotting
Requires-Dist: plotly>=5.0; extra == "plotting"
Requires-Dist: kaleido>=0.2; extra == "plotting"
Provides-Extra: full
Requires-Dist: scikit-learn>=1.3; extra == "full"
Requires-Dist: scipy>=1.11; extra == "full"
Requires-Dist: matplotlib>=3.5; extra == "full"
Requires-Dist: seaborn>=0.12; extra == "full"
Requires-Dist: plotly>=5.0; extra == "full"
Requires-Dist: kaleido>=0.2; extra == "full"
Requires-Dist: umap-learn>=0.5; extra == "full"
Requires-Dist: rapidfuzz>=3.0; extra == "full"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: pytest-cov>=4; extra == "dev"
Requires-Dist: ruff>=0.1; extra == "dev"

# prepx v2

One-line DataFrame cleaning and EDA with advanced imputation, outlier detection, and drift analysis.

```python
from prepx import clean, eda

cleaned_df, clean_report = clean(df)
eda_report = eda(cleaned_df, target="price")
```

---

## Features

### Cleaning (v2 new)
- **Type Coercion** — Auto-convert strings to numeric/datetime BEFORE missing handling
- **Advanced Missing Handling** — ffill, bfill, median/mode, KNN, MICE (iterative imputer)
- **Missing Indicators** — Add binary columns flagging where values were missing
- **Multiple Outlier Methods** — IQR, Z-score, Modified Z-score (MAD), Isolation Forest
- **Outlier Actions** — Cap (winsorize), remove rows, or flag with indicators
- **Fuzzy Deduplication** — rapidfuzz for near-duplicate detection
- **Categorical Normalization** — Standardize "USA" = "U.S.A" = "United States"
- **Leakage Detection** — Flag ID columns, timestamps that cause look-ahead bias

### EDA (v2 new)
- **Distribution Fitting** — Auto-fit 10 distributions, rank by AIC/BIC
- **Multimodality Detection** — Detect hidden subgroups in data
- **Multiple Correlation Methods** — pearson, spearman, kendall
- **VIF (Variance Inflation Factor)** — Detect multicollinearity
- **Drift Detection** — Train/test distribution comparison with PSI, KS test
- **Target Analysis** — Class balance, feature correlations with target

### Architecture
- **Modular** — Import only what you need
- **Optional Dependencies** — Core stays lightweight (~20MB)
- **Backwards Compatible** — v1 API still works

---

## Installation

### Minimal (v1 equivalent)
```bash
pip install prepx
```

### With visualizations
```bash
pip install prepx[viz]
```

### Full features
```bash
pip install prepx[full]
```

Or from source:
```bash
pip install .
```

---

## Dependencies

| Group | Packages | Install size |
|-------|----------|-------------|
| Core | pandas, numpy | ~20MB |
| advanced | sklearn, scipy | +150MB |
| viz | matplotlib, seaborn | +50MB |
| plotting | plotly, kaleido | +30MB |
| full | all above | ~250MB |

---

## `clean(df, **kwargs)` → `(DataFrame, dict)`

Cleans the DataFrame and returns the cleaned version plus a full action report.

### Basic Usage

```python
cleaned, report = clean(df)
```

### Advanced Options

| Parameter | Default | What it does |
|---|---|---|
| `drop_duplicates` | `True` | Remove exact duplicate rows |
| `dedupe_method` | `"exact"` | `"exact"` or `"fuzzy"` |
| `dedupe_threshold` | `0.85` | Similarity threshold for fuzzy |
| `missing_method` | `"auto"` | `"auto"`, `"drop"`, `"ffill"`, `"bfill"`, `"median"`, `"mode"`, `"knn"`, `"mice"` |
| `missing_indicators` | `False` | Add columns flagging where data was missing |
| `missing_threshold` | `0.6` | Drop columns with > this fraction missing |
| `fix_dtypes` | `True` | Coerce strings to numeric/datetime when ≥80% parse |
| `strip_whitespace` | `True` | Strip leading/trailing whitespace |
| `standardize_columns` | `True` | Rename columns to snake_case |
| `naming_style` | `"snake"` | `"snake"`, `"camel"`, `"pascal"`, `"kebab"` |
| `remove_outliers` | `False` | Handle outliers |
| `outlier_method` | `"iqr"` | `"iqr"`, `"zscore"`, `"modified_zscore"`, `"isolation_forest"` |
| `outlier_action` | `"capped"` | `"capped"`, `"removed"`, `"flagged"` |
| `drop_constant_cols` | `True` | Drop columns with only one value |
| `verbose` | `True` | Print report |

### Advanced Examples

```python
# KNN imputation
cleaned, report = clean(df, missing_method="knn")

# Missing indicators (flag where values were missing)
cleaned, report = clean(df, missing_indicators=True)

# MICE (Multiple Imputation by Chained Equations)
cleaned, report = clean(df, missing_method="mice")

# Fuzzy deduplication
cleaned, report = clean(df, dedupe_method="fuzzy", dedupe_threshold=0.85)

# Isolation Forest outliers with flags
cleaned, report = clean(df, remove_outliers=True, outlier_method="isolation_forest", outlier_action="flagged")

# Winsorize (cap extremes)
cleaned, report = clean(df, remove_outliers=True, outlier_action="capped")
```

---

## `eda(df, **kwargs)` → `dict`

Full exploratory analysis.

### Basic Usage

```python
report = eda(df)
```

### Advanced Options

| Parameter | Default | What it does |
|---|---|---|
| `target` | `None` | Target column — adds class balance + correlations |
| `test_data` | `None` | Test DataFrame for drift detection |
| `correlations` | `["pearson"]` | Methods: `"pearson"`, `"spearman"`, `"kendall"` |
| `check_drift` | `False` | Check train/test drift |
| `detect_multimodality` | `False` | Detect bimodal distributions |
| `fit_distributions` | `False` | Fit 10 distributions (requires scipy) |
| `top_n_categories` | `10` | Top categories per column |
| `correlation_threshold` | `0.7` | Threshold for high correlation |
| `verbose` | `True` | Print report |

### Advanced Examples

```python
# With target analysis
report = eda(df, target="churn")

# Multiple correlation methods
report = eda(df, correlations=["pearson", "spearman"])

# Drift detection
train = pd.read_csv("train.csv")
test = pd.read_csv("test.csv")
report = eda(train, test_data=test, check_drift=True)

# Distribution fitting (requires scipy)
report = eda(df, fit_distributions=True)
print(report["distributions"]["income"]["fits"][0]["distribution"])

# Multimodality detection
report = eda(df, detect_multimodality=True)
print(report["multimodality"]["multimodal_columns"])
```

### Report dict keys

| Key | Contents |
|---|---|
| `overview` | Shape, duplicates, memory |
| `dtypes` | Column lists by type |
| `numeric_stats` | Mean, std, min/max, percentiles, skewness, kurtosis, outliers |
| `categorical_stats` | Unique count, top values, mode, entropy |
| `correlations` | Matrix + high-correlation pairs |
| `target_analysis` | Class balance + feature correlations (if target set) |
| `drift_analysis` | PSI/KS scores (if test_data provided) |
| `distributions` | Best-fit distributions (if fit_distributions=True) |
| `warnings` | Auto-generated issues |

---

## Modular API

Import individual functions for more control:

```python
from prepx import (
    coerce_types,
    handle_missing,
    handle_outliers,
    detect_leakage_columns,
    compute_numeric_stats,
    compute_correlations,
    compute_vif,
    compute_drift,
)

# Step-by-step cleaning
df = coerce_types(df)
df, report = handle_missing(df, method="knn")
df, report = handle_outliers(df, method="isolation_forest")

# Modular EDA
stats = compute_numeric_stats(df)
corrs = compute_correlations(df, methods=["pearson", "spearman"])
vif = compute_vif(df)
drift = compute_drift(train_df, test_df)
```

---

## Version History

- **1.0.0** (2025) — Major rewrite with modular architecture
- **0.1.0** — Original release

### Version Contract

- Follows semver: breaking changes only on major bumps
- Deprecated functions kept for one minor version before removal
- Install size kept under 50MB for core

---

## Quick Start

```python
import pandas as pd
from prepx import clean, eda

df = pd.read_csv("data.csv")

# Clean with advanced options
cleaned, clean_report = clean(df, missing_method="knn", remove_outliers=True)

# Full EDA
eda_report = eda(cleaned, target="churn", correlations=["pearson", "spearman"])

# Use reports programmatically
print(eda_report["warnings"])
if eda_report.get("drift_analysis"):
    print("Drift detected:", eda_report["drift_analysis"]["drifted"])
```
