Metadata-Version: 2.4
Name: datacritic
Version: 1.0.0
Summary: Dataset quality analysis and evidence-based cleaning recommendations.
Author: Krish Gupta
License-Expression: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: pandas>=2.0
Requires-Dist: numpy>=1.24

# DataCritic

DataCritic is a Python package for dataset quality analysis and evidence-based cleaning recommendations.

Instead of only detecting missing values or outliers, DataCritic compares possible cleaning strategies and shows how those decisions can affect the observed data.

## Why DataCritic?

Data cleaning is not just about removing bad values.

A cleaning decision can change:

- sample size
- mean
- median
- variance
- standard deviation
- observed distributions
- potential outlier structure

DataCritic helps make those effects visible before a cleaning strategy is applied.

## Features

- Dataset profiling
- Missing-value analysis
- Missing-value strategy comparison
- Outlier detection using the IQR method
- Outlier treatment comparison
- Evidence-based recommendations
- Analysis-aware recommendation modes
- Explainable reasoning
- Impact metrics
- Audit-style recommendation reports
- Edge-case validation

## Installation

```bash
pip install datacritic
```

## Quick Start

```python
import pandas as pd

from datacritic import DataCritic

df = pd.DataFrame({
    "age": [21, 22, 23, None, 25],
    "salary": [30000, 32000, 31000, 35000, 1000000],
    "city": [
        "Delhi",
        "Mumbai",
        "Delhi",
        None,
        "Mumbai",
    ],
})

critic = DataCritic(df)
```

## Dataset Profile

```python
profile = critic.profile()

print(profile)
```

The profile includes:

- number of rows
- number of columns
- data types
- missing values
- missing percentages
- duplicate rows
- constant columns

## Missingness Analysis

```python
result = critic.missingness()

print(result)
```

This identifies columns containing missing values and reports their missing-value counts and percentages.

## Outlier Analysis

```python
result = critic.outliers()

print(result)
```

DataCritic currently uses the IQR method for numeric outlier detection.

## Outlier Treatment Comparison

For numeric columns, DataCritic can compare:

```python
critic.outlier_candidates("salary")
```

Available strategies:

```text
keep
remove_rows
clip
```

Impact can be evaluated with:

```python
critic.outlier_impact(
    "salary",
    "clip",
)
```

Or all strategies can be compared:

```python
critic.compare_outliers("salary")
```

## Missing-Value Strategies

DataCritic can compare:

```text
drop_rows
drop_column
mean
median
mode
```

For example:

```python
critic.compare("salary")
```

The comparison can show changes in:

- retained rows
- missing values
- mean
- median
- standard deviation
- variance

## Recommendations

DataCritic provides recommendations based on observed dataset characteristics.

```python
result = critic.recommend(
    "salary",
    task="exploration",
)

print(result)
```

Supported analysis objectives:

```text
exploration
prediction
reporting
```

These objectives provide context for the recommendation and its explanation. They do not replace validation within a downstream analysis or machine-learning pipeline.

## Explainable Reports

Generate a readable recommendation report:

```python
report = critic.recommendation_report(
    "salary"
)

report.show()
```

The report includes:

- detected dataset issues
- recommended strategy
- strategy comparisons
- outlier impact
- evidence
- reasoning
- limitations

## Example Report

```text
============================================================
                    DATACRITIC REPORT
============================================================

DATASET ISSUE
------------------------------------------------------------
Column          : salary
Task            : exploration
Missing values  : 1
Missing percent : 12.50%
Outliers        : Detected 1 potential outlier(s).

RECOMMENDED STRATEGY
------------------------------------------------------------
MEDIAN

STRATEGY COMPARISON
------------------------------------------------------------
...

OUTLIER IMPACT
------------------------------------------------------------
...

EVIDENCE
------------------------------------------------------------
...

REASONING
------------------------------------------------------------
...

LIMITATIONS
------------------------------------------------------------
...
```

## Design Philosophy

DataCritic follows a simple principle:

> Don't just clean the data. Evaluate what you may change by cleaning it.

The package is designed to make cleaning decisions more transparent by exposing measurable changes produced by candidate strategies.

## Limitations

Recommendations are based on observed dataset characteristics.

DataCritic does not claim to determine the true missingness mechanism from observed data alone.

Recommendations should be validated against the requirements of the specific dataset, analysis, and downstream model.

For predictive modelling, preprocessing should be evaluated inside the appropriate training and validation pipeline.

## Testing

The project uses pytest.

Run the test suite with:

```bash
pytest -q
```

The test suite covers:

- dataset validation
- profiling
- missingness analysis
- strategy generation
- impact analysis
- outlier detection
- outlier treatment
- recommendations
- reporting
- edge cases

## Project Structure

```text
DataCritic/
├── examples/
├── src/
│   └── datacritic/
│       ├── __init__.py
│       ├── critic.py
│       ├── impact.py
│       ├── missingness.py
│       ├── outliers.py
│       ├── profiler.py
│       ├── recommendation.py
│       ├── report.py
│       └── strategies.py
├── tests/
├── README.md
├── pyproject.toml
└── .gitignore
```

## License

MIT License.
