Metadata-Version: 2.4
Name: textprep-lite
Version: 0.1.0
Summary: Lightweight text cleaning and anonymization for real-world NLP
Author: Your Name
License: MIT
Project-URL: Homepage, https://github.com/YOUR_USERNAME/textprep-lite
Project-URL: Source, https://github.com/YOUR_USERNAME/textprep-lite
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Text Processing :: Filters
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# textprep-lite

**Lightweight text cleaning and anonymization for real-world NLP.**

`textprep-lite` is a tiny Python library that standardizes text normalization and anonymization in 1–2 function calls, designed for ML/NLP workflows.

## Installation

```bash
pip install textprep-lite
```

## Quickstart

```python
from textprep_lite import clean_text, anonymize_text, preprocess_text

raw = "Hey!!   <br> This is  John   \nContact: john.doe@example.com, +44 7700 900123"

# Basic cleaning
cleaned = clean_text(raw)
print(cleaned)
# Output: "hey!! this is john contact: john.doe@example.com, +44 7700 900123"

# Anonymization
anon = anonymize_text(raw)
print(anon)
# Output: "Hey!!   <br> This is  John   \nContact: [REDACTED], [REDACTED]"

# Full pipeline
processed = preprocess_text(raw)
print(processed)
# Output: "hey!! this is john contact: [redacted], [redacted]"
```

## Configuration

### `clean_text`

| Argument | Default | Description |
| :--- | :--- | :--- |
| `lowercase` | `True` | Convert text to lowercase. |
| `strip_html` | `True` | Remove HTML tags. |
| `normalize_whitespace` | `True` | Collapse multiple spaces/newlines. |
| `strip_accents` | `False` | Normalize Unicode accents to ASCII. |
| `remove_punct` | `False` | Remove punctuation. |
| `remove_digits` | `False` | Remove digits. |
| `keep_newlines` | `False` | Preserve newlines during whitespace normalization. |

### `anonymize_text`

| Argument | Default | Description |
| :--- | :--- | :--- |
| `mask_emails` | `True` | Mask email addresses. |
| `mask_phones` | `True` | Mask phone numbers. |
| `mask_urls` | `True` | Mask URLs. |
| `mask_ip` | `False` | Mask IPv4 addresses. |
| `replacement` | `"[REDACTED]"` | Replacement string for masked items. |

## Why `textprep-lite`?

Real-world text is messy. You repeatedly need to normalize whitespace, strip HTML, and mask PII. Instead of copying the same regex snippets into every notebook, `textprep-lite` offers a tested, reusable, and composable way to clean your text data.

## Testing & Quality

`textprep-lite` is designed for reliability.
- **Unit Tests**: Comprehensive coverage for all cleaning and anonymization logic.
- **Stress Tested**: Verified against edge cases (empty strings, emojis, mixed IP/phone formats) and synthetic logs.
- **Clean API**: Type-hinted and documented.

## Contributing

1. Clone the repo.
2. Install dependencies (e.g., `pip install -e .[dev]`).
3. Run tests:
    ```bash
    pytest
    ```
