Metadata-Version: 2.4
Name: dd-parser-cleaner
Version: 0.7.0
Summary: A private, local LLM-powered data dictionary parser and entity mapper with automated cleaning.
Project-URL: Homepage, https://github.com/user/dd-parser-cleaner
Project-URL: Repository, https://github.com/user/dd-parser-cleaner
Author-email: Rajiv <rajiv.sambasivan@r2-ds.com>
License: MIT
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27.0
Requires-Dist: ollama>=0.2.0
Requires-Dist: pandas>=2.2.0
Requires-Dist: pydantic>=2.6.0
Requires-Dist: pypdf>=4.1.0
Requires-Dist: pyyaml>=6.0.1
Requires-Dist: requests>=2.34.2
Requires-Dist: rich>=13.7.0
Requires-Dist: tabulate>=0.10.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Provides-Extra: notebook
Requires-Dist: ipykernel>=6.29.0; extra == 'notebook'
Requires-Dist: jupyter>=1.1.1; extra == 'notebook'
Requires-Dist: notebook>=7.5.6; extra == 'notebook'
Description-Content-Type: text/markdown

# dd-parser-cleaner

A lightweight framework for documenting and automating dataset preparation with AI-driven metadata discovery.

## What it does
- Builds a documented cleaning workflow for raw datasets.
- Creates structured metadata and transformation guidance for later featurization.
- Supports repeatable, deterministic dataset preparation in local environments.

## Real-world examples
For complete, real-world migration examples, see:
- SBA dataset: https://github.com/rajivsam/kmds_migration/blob/main/sba_migration/documents/sba_development_example_full_doc.md
- Olist dataset: https://github.com/rajivsam/kmds_migration/blob/main/olist_migration/documents/olist_development_example_full_doc.md

These examples show how the tool is applied to cross-sectional datasets. An example for longitudinal/panel datasets is coming soon.

## Why it matters
- Saves time by automating metadata discovery and documentation.
- Keeps data cleaning transparent and audit-ready.
- Makes downstream featurization easier because the cleaning process is already documented.

## Quick start

### Install
```bash
pip install dd-parser-cleaner
```

### Initialize a workspace
```bash
uv run init-workspace ./my_project
uv run bootstrap-config ./my_project
```

### Discover package features from Python
```python
from dd_parser_cleaner import get_package_info

info = get_package_info()
print(info)
```

### Run the core workflow
```bash
classify-entities
uv run clean-dataset --action full --workspace ./my_project
```

## Where to look next
- `USER_GUIDE.md` for usage details
- `documents/` for methodology and internal design notes
- `tests/notebooks/` for example notebook workflows
