Metadata-Version: 2.4
Name: frameprep
Version: 0.1.0
Summary: Automated ML-Ready Data Preparation Library
Author-email: Rahul Reddy <rahulreddy9725@gmail.com>
License: MIT License
        
        Copyright (c) 2026 Rahul Reddy
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/rahulreddy9725/frameprep
Project-URL: Documentation, https://github.com/rahulreddy9725/frameprep/blob/main/README.md
Project-URL: Bug Tracker, https://github.com/rahulreddy9725/frameprep/issues
Project-URL: Changelog, https://github.com/rahulreddy9725/frameprep/blob/main/CHANGELOG.md
Keywords: machine learning,data preparation,preprocessing,feature engineering,data science,imputation,encoding,leakage detection
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: scikit-learn>=1.3
Requires-Dist: scipy>=1.10
Provides-Extra: dev
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: pytest-cov>=4.1; extra == "dev"
Requires-Dist: ruff>=0.1; extra == "dev"
Requires-Dist: mypy>=1.5; extra == "dev"
Requires-Dist: pandas-stubs>=2.0; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=7.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=1.3; extra == "docs"
Requires-Dist: myst-parser>=2.0; extra == "docs"
Dynamic: license-file

# frameprep

**Automated ML-Ready Data Preparation Library**

> One pipeline. Raw DataFrame in. ML-ready DataFrame out.

---

## The Problem

Data teams at mid-size companies spend **60–70 % of project time** cleaning and
preparing raw data before any model can touch it. `frameprep` automates the
most repetitive and error-prone parts of that work.

---

## What It Does

| Module | What It Handles |
|--------|----------------|
| **Smart dtype inference** | Detects real types (bool stored as string, datetime in text, numeric strings) and downcasts to smallest memory-safe type |
| **Missing value strategy** | Runs statistical tests (MCAR vs MAR proxy) to pick the right imputer per column — mean, median, KNN, mode, or sentinel |
| **Encoding + scaling** | Label, OHE, or hashing for categoricals; standard / minmax / robust for numerics |
| **Leakage detection + audit** | Flags correlated, near-constant, and ID-like columns; produces a full JSON/DataFrame audit report |

---

## Installation

```bash
pip install frameprep
```

For development:

```bash
git clone  https://github.com/rahulreddy9725/frameprep.git
cd frameprep
pip install -e ".[dev]"
```

---

## Quick Start

```python
import pandas as pd
from frameprep import frameprepPipeline

df_raw = pd.read_csv("customers.csv")

pipeline = frameprepPipeline(
    target_col="churn",          # excluded from encoding/scaling
    scaling_strategy="standard", # 'standard' | 'minmax' | 'robust'
    correlation_threshold=0.95,  # leakage flag threshold
    verbose=True,
)

df_clean, report = pipeline.fit_transform(df_raw)

# Inspect the audit
report.summary()                 # prints to stdout
report.to_json()                 # full JSON string
report.to_dataframe()            # flat pandas DataFrame
report.save("audit.json")        # write to disk

# Apply the same transforms to test/prod data
df_test_clean = pipeline.transform(df_test)
```

---

## Audit Report Example

```
============================================================
  frameprep — Preparation Audit Report
============================================================
  Created at   : 2024-01-15T10:23:45+00:00
  Elapsed      : 0.83s

  Shape
    Input  : 5000 rows × 18 columns
    Output : 5000 rows × 24 columns
    Δ cols : +6

  Memory Optimisation
    Before : 720.0 KB
    After  : 218.4 KB
    Saved  : 69.7%

  Imputation Strategies
    median              : 4 column(s)
    knn                 : 2 column(s)
    mode                : 3 column(s)
    none                : 9 column(s)

  Encoding Strategies
    one_hot_encode      : 3 column(s)
    label_encode        : 2 column(s)
    feature_hash        : 1 column(s)

  Scaling Strategy   : standard (8 columns)

  Leakage Detected   : YES ⚠
    • row_id: monotonically increasing integer — likely a row ID
============================================================
```

---

## Configuration Reference

```python
frameprepPipeline(
    target_col=None,              # str  — label column, excluded from transforms
    exclude_cols=[],              # list — columns to pass through unchanged
    correlation_threshold=0.95,  # float — leakage correlation cutoff
    cardinality_threshold=50,    # int  — above this → hashing, below → OHE
    scaling_strategy="standard", # str  — 'standard' | 'minmax' | 'robust'
    verbose=True,                # bool — step-by-step logging
)
```

---

## Module Details

### 1. DTypeInferrer

- Detects boolean strings (`"yes"/"no"`, `"true"/"false"`)
- Detects datetime strings (`"2023-01-15"`, `"15/01/2023"`)
- Detects numeric strings (`"100"`, `"3.14"`)
- Downcasts `float64 → float32`, `int64 → int8/int16/int32`
- Low-cardinality objects → `pd.Categorical`

### 2. MissingValueHandler

| Condition | Strategy |
|-----------|----------|
| 0 % missing | skip |
| < 1 % | median / mode (fast) |
| MAR detected (missingness correlated with other columns) | KNN imputer |
| \|skew\| > 1 | median |
| \|skew\| ≤ 1 | mean |
| > 40 % missing | constant flag (-999) |
| Categorical < 30 % | mode |
| Categorical ≥ 30 % | `"__missing__"` sentinel |

### 3. CategoricalEncoder

| Cardinality | Strategy |
|-------------|----------|
| 2 unique values | LabelEncoder |
| 3 – threshold | OneHotEncoder (drop first) |
| > threshold | FeatureHasher |
| Boolean dtype | cast to 0/1 |

### 4. LeakageDetector

Runs three checks:

- **Near-constant**: > 99 % of values identical
- **High correlation with target**: Pearson |r| ≥ threshold
- **ID-like column**: monotonically increasing integer

---

## Development

```bash
# Run tests
pytest

# Run tests with coverage
pytest --cov=frameprep --cov-report=term-missing

# Lint
ruff check frameprep tests

# Type check
mypy frameprep
```

---

## Project Structure

```
frameprep/
├── frameprep/
│   ├── __init__.py
│   ├── pipeline.py          # Main frameprepPipeline
│   ├── core/
│   │   ├── dtype_inferrer.py
│   │   └── missing_strategy.py
│   ├── encoders/
│   │   └── categorical.py
│   ├── scalers/
│   │   └── numeric.py
│   ├── detectors/
│   │   └── leakage.py
│   ├── reports/
│   │   └── audit.py
│   └── utils/
│       ├── logger.py
│       └── validators.py
├── tests/
│   ├── conftest.py
│   ├── unit/
│   │   ├── test_dtype_inferrer.py
│   │   ├── test_missing_strategy.py
│   │   ├── test_categorical_encoder.py
│   │   ├── test_numeric_scaler.py
│   │   └── test_leakage_detector.py
│   └── integration/
│       └── test_pipeline.py
├── docs/examples/
│   └── quickstart.ipynb
├── pyproject.toml
├── README.md
├── CONTRIBUTING.md
├── CHANGELOG.md
└── LICENSE
```

---

## License

MIT — see [LICENSE](LICENSE).
