Metadata-Version: 2.4
Name: klino
Version: 1.0.1
Summary: Fast data cleaning CLI for ML, NLP, CV and Time Series
License-Expression: MIT
Project-URL: Homepage, https://github.com/Vikas8719/klino
Project-URL: Repository, https://github.com/Vikas8719/klino
Project-URL: Issues, https://github.com/Vikas8719/klino/issues
Project-URL: Changelog, https://github.com/Vikas8719/klino/releases
Keywords: data-cleaning,cli,ml,nlp,cv,time-series
Classifier: Programming Language :: Python :: 3.11
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.1
Requires-Dist: rich>=13.0
Requires-Dist: pandas>=2.1
Requires-Dist: polars-lts-cpu==1.9.0
Requires-Dist: openpyxl>=3.1
Requires-Dist: pillow>=10.0
Requires-Dist: opencv-python-headless>=4.8
Requires-Dist: beautifulsoup4>=4.12
Requires-Dist: lxml>=5.0
Requires-Dist: nltk>=3.8
Requires-Dist: requests>=2.31
Requires-Dist: pyarrow
Dynamic: license-file

# Klino — Fast Data Cleaning CLI for ML, NLP, CV & Time Series

<p align="center">
  <img src="https://img.shields.io/pypi/v/klino?color=blue&label=PyPI" alt="PyPI version">
  <img src="https://img.shields.io/pypi/pyversions/klino" alt="Python versions">
  <img src="https://img.shields.io/pypi/l/klino" alt="License">
  <img src="https://img.shields.io/pypi/dm/klino?color=green" alt="Downloads">
</p>

> **One command. Clean data.**  
> Klino is a zero-config command-line tool that prepares raw datasets for machine learning — handling CSV, Excel, text, HTML, images, and time series with a single command.

---

## Why Klino?

Data cleaning is the most time-consuming step in any ML project. Klino automates the full pipeline:

- **Scan first, clean second** — inspect every problem before touching a single row
- **Auto-detects file type** — no flags, no config files
- **Industry-standard algorithms** — IQR + Z-score hybrid outlier removal, ADF stationarity test, WordNet lemmatization, letterbox pad-resize
- **Two engines** — pandas (free, < 50 MB) and polars ⚡ (Pro, any size) with automatic fallback
- **Non-destructive** — original file is never modified; output is always saved as `clean_<filename>` in the same folder

---

## Installation

```bash
pip install klino
```

Requires Python ≥ 3.11.

---

## Quick Start

```bash
# 1. Scan your data — see every problem before cleaning
klino scan data.csv

# 2. Clean it — one command does everything
klino ml data.csv

# 3. Verify — scan the output to confirm all issues are gone
klino scan clean_data.csv
```

Output is saved as **`clean_data.csv`** in the same folder as the original file.

---

## Commands

### `klino scan` — Inspect Data (Read-Only)

Scans your dataset and reports every issue in a rich table. **Nothing is modified.**  
Run it before cleaning to see how much work needs to be done, and again after cleaning to verify the output is clean.

```bash
klino scan data.csv           # ML scan  — CSV / Excel
klino scan corpus.txt         # NLP scan — text / HTML (auto-detected)
klino scan ./images/          # CV scan  — image folder (auto-detected)
klino scan-ts sales.csv       # TS scan  — time series (explicit)
klino scan-nlp reviews.html   # NLP scan — explicit
klino scan-cv ./train/        # CV scan  — explicit
```

**ML scan checks:** missing values (per column), duplicate rows, constant columns, outlier rows (IQR + Z-score), category inconsistencies, whitespace issues.

**NLP scan checks:** HTML tags, URLs, @mentions, #hashtags, emojis, contractions, extra whitespace, duplicate sentences.

**CV scan checks:** corrupt / unreadable images, duplicate images (MD5 hash), images below 32×32 px, too-dark images (brightness < 20), overexposed images (brightness > 240), inconsistent sizes.

**TS scan checks:** missing values, duplicate timestamps, future data leakage, irregular frequency, spike / outlier rows (rolling Z-score), non-numeric columns.

---

### `klino ml` — Clean CSV / Excel for ML

```bash
klino ml data.csv                        # auto engine
klino ml data.csv --engine pandas        # force pandas
klino ml data.csv --engine polars        # force polars ⚡
klino ml data.csv -o /path/to/output.csv # custom output path
klino ml                                 # scans current folder for CSV/Excel
```

**Pipeline (in order):**
1. Auto-fix data types (numeric, datetime coercion)
2. Remove duplicate rows
3. Drop constant columns (zero variance)
4. Fill missing values — numeric → median, categorical → mode
5. Remove outliers — IQR (Tukey's fence) ∩ Z-score (3σ) intersection
6. Normalise category values — strip whitespace, title-case
7. Clean text columns — collapse whitespace

---

### `klino nlp` — Clean Text / HTML for NLP

```bash
klino nlp corpus.txt
klino nlp page.html --lang english
klino nlp tweets.csv -o clean_tweets.txt
```

**Pipeline (in order):**
1. Strip HTML tags and decode entities
2. Expand contractions (`don't` → `do not`)
3. Remove emojis
4. Remove URLs, @mentions, #hashtags
5. Lowercase
6. Remove punctuation
7. Remove stopwords
8. Lemmatize (WordNet — verbs + nouns)
9. Remove duplicate sentences

---

### `klino cv` — Clean Image Folders for CV / Deep Learning

```bash
klino cv ./images/
klino cv ./images/ --size 224x224       # custom target size (default: 224×224)
klino cv ./images/ -o ./cleaned/        # custom output folder
```

**Pipeline (in order):**
1. Skip corrupt / unreadable images
2. Skip images below minimum dimension (32×32 px)
3. Deduplicate by MD5 hash of raw bytes
4. Filter by brightness — skip too-dark (< 20) and overexposed (> 240)
5. Aspect-ratio–preserving letterbox pad-resize to target size
6. Normalise pixel values to float32 `[0, 1]`
7. Save as `.npy` files (compatible with NumPy, PyTorch, TensorFlow)

---

### `klino ts` — Clean Time Series Data

```bash
klino ts sensor.csv
klino ts logs.xlsx --engine polars
klino ts data.csv -o clean_data.csv
```

**Pipeline (in order):**
1. Detect and parse datetime column, convert to UTC
2. Remove duplicate timestamps
3. Remove future rows (data leakage prevention)
4. Auto-resample to regular frequency (inferred from median gap)
5. Fill missing values via time-aware interpolation → ffill → bfill
6. Remove spikes — rolling Z-score per column (window=5, σ > 3.0)
7. Noise smoothing — rolling mean (window=3)
8. Stationarity check — ADF test + first-order differencing per column

---

### License Commands

```bash
klino activate YOUR-LICENSE-KEY     # activate Klino Pro (1 machine per key)
klino deactivate                    # release key to move to a new machine
klino status                        # show tier, expiry, days remaining
klino --version                     # show installed version
```

---

## Free vs Pro

| Feature | Free | Pro |
|---|---|---|
| File size limit | < 50 MB | Unlimited |
| Engine | pandas | polars ⚡ (CPU-parallel) |
| All cleaning steps | ✅ | ✅ |
| Files 50–200 MB | ❌ | ✅ Full pipeline |
| Files ≥ 200 MB | ❌ | ✅ Fast mode (safe steps) |
| Offline grace period | ❌ | ✅ 7 days |
| Machines per key | — | 1 (transferable) |

**Pro license:** [https://vikasawasti.gumroad.com/l/klinopro](https://vikasawasti.gumroad.com/l/klinopro)  
Pay → key emailed instantly → `klino activate YOUR-KEY`

---

## Output Files

All cleaned files are saved in the **same folder as the input file**, prefixed with `clean_`:

| Input | Output |
|---|---|
| `data.csv` | `clean_data.csv` |
| `corpus.txt` | `clean_corpus.txt` |
| `page.html` | `clean_page.txt` |
| `sensor.xlsx` | `clean_sensor.xlsx` |
| `./images/` | `./images_clean/` (sibling folder) |

---

## Engines

| Engine | Speed | Memory | When |
|---|---|---|---|
| pandas | Baseline | Standard | Files < 50 MB (free tier) |
| polars ⚡ | 3–8× faster | Lower | Files ≥ 50 MB (Pro) |

Engine is selected automatically based on file size. Override with `--engine pandas` or `--engine polars`.

---

## Requirements

- Python ≥ 3.11
- Dependencies are installed automatically: `pandas`, `polars`, `rich`, `click`, `nltk`, `beautifulsoup4`, `opencv-python-headless`, `pillow`, `openpyxl`, `lxml`, `requests`
- No GPU required — all processing is CPU-only

---

## License

MIT — see [LICENSE](LICENSE)
