Metadata-Version: 2.4
Name: smartclean-ai
Version: 1.2.3
Summary: An intelligent Python library for automated structured-data cleaning, profiling, outlier handling, and reporting.
Author-email: Yash Talati <yashtalati07@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/yashtalati10/SmartClean
Project-URL: Repository, https://github.com/yashtalati10/SmartClean
Project-URL: Issues, https://github.com/yashtalati10/SmartClean/issues
Keywords: data-cleaning,automated-data-cleaning,data-quality,data-profiling,exploratory-data-analysis,missing-values,outlier-detection,pandas,numpy,preprocessing,csv,excel,python
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=2.0.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: matplotlib>=3.7.0
Requires-Dist: scikit-learn>=1.3.0
Requires-Dist: openpyxl>=3.1.0
Requires-Dist: xlrd>=2.0.1
Dynamic: license-file

# SmartClean-AI

**Automated Structured-Data Cleaning and Reporting in Python**

SmartClean-AI is a Python library designed to help prepare structured datasets for analysis by addressing common data-quality issues and providing access to cleaning reports and dataset profiles.

**Project links**

- **PyPI:** https://pypi.org/project/smartclean-ai/
- **GitHub:** https://github.com/yashtalati10/SmartClean
- **Issues and bug reports:** https://github.com/yashtalati10/SmartClean/issues

---

## Table of Contents

1. Overview
2. Features
3. Installation
4. Requirements
5. Quick Start
6. Complete Output Boilerplate
7. Understanding the Output Files
8. Working with CleaningResult
9. Input Data
10. Example Project Structure
11. Troubleshooting
12. Contributing
13. License

---

## 1. Overview

Real-world datasets may contain missing values, duplicate records, inconsistent data, and outliers. These issues can complicate exploratory data analysis (EDA) and downstream machine-learning workflows.

SmartClean-AI provides a Python interface for structured-data cleaning and exposes cleaning results, reports, and dataset profiles.

The library is intended to help users prepare data more efficiently while keeping the cleaned dataset and report files available for inspection.

## 2. Features

- **Missing-value handling:** Handle missing values through the implemented cleaning pipeline.
- **Duplicate handling:** Support duplicate-record handling where configured in the pipeline.
- **Outlier handling:** Support outlier processing through the implemented handler and selected method.
- **Dataset profiles:** Access before-cleaning and after-cleaning profiles through `CleaningResult`.
- **Cleaning summaries:** Display a formatted summary using `display_report()`.
- **CSV export:** Save the cleaned DataFrame using `save_csv()`.
- **JSON reports:** Save structured cleaning information using `save_report()`.
- **TXT reports:** Export a readable report using `save_report()`.
- **Custom reporting:** Use the result object to build additional reports and visualizations.

The exact cleaning behavior depends on the installed version and the pipeline implementation.

## 3. Installation

Install the package from PyPI:

```bash
python -m pip install smartclean-ai
```

Check the installed version:

```bash
python -m pip show smartclean-ai
```

Import the public API:

```python
from smartclean import Cleaner, CleaningResult

print("SmartClean-AI imports successfully.")
```

## 4. Requirements

The project metadata specifies these dependencies:

- Python 3.10 or later
- NumPy
- pandas
- Matplotlib
- scikit-learn

Install or upgrade the dependencies through pip if needed:

```bash
python -m pip install --upgrade numpy pandas matplotlib scikit-learn
```

## 5. Quick Start

The following example illustrates the intended workflow. Confirm the constructor and cleaning method against the installed `smartclean/cleaner.py` implementation before running it.

```python
from smartclean import Cleaner

# Path to your input dataset
input_file = "data.csv"

# Initialize the cleaner
cleaner = Cleaner(input_file)

# Run the cleaning workflow
result = cleaner.clean()

# Display the cleaning report
result.display_report()

# Save the cleaned dataset
result.save_csv("cleaned_data.csv")

# Save JSON and TXT reports
result.save_report("cleaning_report.json", format="json")
result.save_report("cleaning_report.txt", format="txt")

print("Cleaning and export workflow completed.")
```

If your installed version uses a different method to start the cleaning workflow, replace `cleaner.clean()` with the actual method implemented by `Cleaner`.

## 6. Complete Output Boilerplate

This example shows how a user can organize a single run to produce four deliverables:

1. Cleaned CSV
2. JSON cleaning report
3. TXT cleaning report
4. Dashboard PNG

The CSV and reports use the `CleaningResult` methods shown in the project source. The dashboard section is an example integration and requires a compatible dashboard function.

Save the following as `run_smartclean.py` after verifying the cleaning call and dashboard function signature.

```python
from pathlib import Path
import json
import pandas as pd
import matplotlib.pyplot as plt

from smartclean import Cleaner

# --------------------------------------------------
# CONFIGURATION
# --------------------------------------------------

INPUT_FILE = Path("data.csv")
OUTPUT_DIR = Path("smartclean_output")

OUTPUT_DIR.mkdir(parents=True, exist_ok=True)

CSV_OUTPUT = OUTPUT_DIR / "cleaned_data.csv"
JSON_OUTPUT = OUTPUT_DIR / "cleaning_report.json"
TXT_OUTPUT = OUTPUT_DIR / "cleaning_report.txt"
DASHBOARD_OUTPUT = OUTPUT_DIR / "cleaning_dashboard.png"


# --------------------------------------------------
# RUN CLEANING
# --------------------------------------------------

def main():
    if not INPUT_FILE.exists():
        raise FileNotFoundError(
            f"Input dataset was not found: {INPUT_FILE.resolve()}"
        )

    print(f"Input dataset: {INPUT_FILE.resolve()}")

    # Verify this call against the installed Cleaner API.
    cleaner = Cleaner(str(INPUT_FILE))
    result = cleaner.clean()

    # --------------------------------------------------
    # 1. DISPLAY REPORT
    # --------------------------------------------------

    result.display_report()

    # --------------------------------------------------
    # 2. EXPORT CLEANED CSV
    # --------------------------------------------------

    result.save_csv(str(CSV_OUTPUT))

    # --------------------------------------------------
    # 3. EXPORT JSON REPORT
    # --------------------------------------------------

    result.save_report(str(JSON_OUTPUT), format="json")

    # --------------------------------------------------
    # 4. EXPORT TXT REPORT
    # --------------------------------------------------

    result.save_report(str(TXT_OUTPUT), format="txt")

    # --------------------------------------------------
    # 5. GENERATE A BASIC DATASET DASHBOARD
    # --------------------------------------------------

    cleaned_df = result.cleaned_df
    before_df = pd.read_csv(INPUT_FILE)

    missing_before = int(before_df.isna().sum().sum())
    missing_after = int(cleaned_df.isna().sum().sum())

    rows_before = len(before_df)
    rows_after = len(cleaned_df)

    columns_before = len(before_df.columns)
    columns_after = len(cleaned_df.columns)

    fig, axes = plt.subplots(1, 2, figsize=(11, 5))

    # Dataset dimensions
    axes[0].bar(
        ["Before", "After"],
        [rows_before, rows_after],
        label="Rows"
    )
    axes[0].set_title("Dataset Row Count")
    axes[0].set_ylabel("Number of rows")

    # Missing-value counts
    axes[1].bar(
        ["Before", "After"],
        [missing_before, missing_after],
        label="Missing values"
    )
    axes[1].set_title("Missing Values")
    axes[1].set_ylabel("Number of missing values")

    fig.suptitle("SmartClean-AI Cleaning Dashboard")
    fig.tight_layout()
    fig.savefig(DASHBOARD_OUTPUT, dpi=160, bbox_inches="tight")
    plt.close(fig)

    # --------------------------------------------------
    # COMPLETION SUMMARY
    # --------------------------------------------------

    print("\nAll requested output files have been generated:")
    print(f"Cleaned CSV:    {CSV_OUTPUT.resolve()}")
    print(f"JSON report:    {JSON_OUTPUT.resolve()}")
    print(f"TXT report:     {TXT_OUTPUT.resolve()}")
    print(f"Dashboard PNG:  {DASHBOARD_OUTPUT.resolve()}")


if __name__ == "__main__":
    main()
```

### Important notes about the boilerplate

- The example assumes `Cleaner(input_path)` and `cleaner.clean()` are supported. Verify those two calls before publishing this as a guaranteed runnable example.
- The dashboard above is generated by the example script using Matplotlib. It is **not represented as a built-in `CleaningResult` dashboard method**.
- This example reads the input with `pandas.read_csv()`. For Excel input, the dashboard's input-loading line must be adapted to the supported Excel format.
- The JSON and TXT report methods serialize the report and available profiles; they do not automatically include arbitrary dashboard metrics unless those metrics are part of the stored report.
- The unused `json` import can be removed from this example; the report export is handled by `result.save_report()`.

## 7. Understanding the Output Files

After a successful run, the output directory should look like this:

```text
smartclean_output/
├── cleaned_data.csv
├── cleaning_report.json
├── cleaning_report.txt
└── cleaning_dashboard.png
```

| File | Purpose |
|---|---|
| `cleaned_data.csv` | Contains the cleaned dataset. |
| `cleaning_report.json` | Stores structured report and profile information. |
| `cleaning_report.txt` | Stores a human-readable report. |
| `cleaning_dashboard.png` | Visualizes selected before-and-after dataset statistics. |

The dashboard values should be interpreted alongside the report. Fewer missing values or fewer rows do not automatically mean every dataset is better; review the transformations before using the results.

## 8. Working with CleaningResult

The public package exports `CleaningResult`, which provides access to the cleaned dataset, profiles, and cleaning report.

### Access the cleaned DataFrame

```python
cleaned_df = result.cleaned_df

print(cleaned_df.head())
print(cleaned_df.shape)
```

### Display the report

```python
result.display_report()
```

### Save the cleaned dataset

```python
result.save_csv("output/cleaned_data.csv")
```

### Save a JSON report

```python
result.save_report("output/report.json", format="json")
```

### Save a TXT report

```python
result.save_report("output/report.txt", format="txt")
```

### Inspect the report and profiles

```python
print(result.report)
print(result.before_profile)
print(result.after_profile)
```

These objects can be used to build custom summaries and visualizations. The exact keys and nested structure depend on the reports returned by the implemented cleaning handlers.

## 9. Input Data

Prepare a CSV or Excel file supported by the installed data loader and pass its path to the cleaning workflow.

Example CSV:

```csv
Name,Age,City,Salary
Aarav,25,Pune,45000
Meera,,Mumbai,52000
Kabir,32,Pune,
Aarav,25,Pune,45000
```

This small dataset illustrates missing values and a repeated record. It is an illustrative input, not a guarantee of how every cleaning setting will transform the rows.

For Excel input, use an extension and loading method supported by the version you have installed.

## 10. Example Project Structure

```text
my-data-project/
├── data.csv
├── run_smartclean.py
└── smartclean_output/
    ├── cleaned_data.csv
    ├── cleaning_report.json
    ├── cleaning_report.txt
    └── cleaning_dashboard.png
```

Keep the original input file unchanged so that you can compare the original dataset with the output.

## 11. Troubleshooting

### Import error

```text
ModuleNotFoundError: No module named 'smartclean'
```

Install the package in the active Python environment:

```bash
python -m pip install smartclean-ai
```

### Input file not found

Check the path and filename before running the script. Relative paths are resolved from the script's current working directory.

### Report method or attribute error

Confirm the installed package version and inspect the `Cleaner` and `CleaningResult` APIs. Do not assume a method exists unless it is implemented in that version.

### Excel loading error

Check whether the file extension is supported and whether the necessary Excel-reading dependency is installed.

### Dashboard generation error

Ensure Matplotlib is installed and that the dashboard code uses numeric statistics available in the selected dataset.

## 12. Contributing

Contributions, bug reports, and feature suggestions are welcome.

1. Fork the repository.
2. Create a branch for your change.
3. Test your changes locally.
4. Submit a pull request with a description of the change.

Repository: https://github.com/yashtalati10/SmartClean

## 13. License

This project declares the MIT license in its package metadata. Refer to the repository's license file for the complete license terms.
