Metadata-Version: 2.2
Name: ragkit-redactor
Version: 0.1.0
Summary: Detect and reversibly mask PII / secrets in text before sending it to an LLM.
Author-email: Meet2147 <meetjethwa3@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/Meet2147/pythonLibraries/tree/main/redactor
Project-URL: Repository, https://github.com/Meet2147/pythonLibraries
Project-URL: Issues, https://github.com/Meet2147/pythonLibraries/issues
Keywords: pii,redaction,secrets,llm,privacy,masking,genai
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Filters
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE

<p align="center">
  <img src="https://raw.githubusercontent.com/Meet2147/pythonLibraries/main/redactor/assets/logo.png" alt="redactor" width="460">
</p>

<p align="center">
  <a href="https://pypi.org/project/ragkit-redactor/"><img src="https://img.shields.io/pypi/v/ragkit-redactor.svg" alt="PyPI"></a>
  <img src="https://img.shields.io/pypi/pyversions/ragkit-redactor.svg" alt="Python versions">
  <img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License: MIT">
</p>

# redactor

**Detect and reversibly mask PII / secrets in text before you send it to an LLM — then un-mask the model's reply.**

> Part of the **ragkit** suite. Install with `pip install ragkit-redactor`, then `import redactor`.

`redactor` scans text for things you probably don't want leaving your machine —
emails, phone numbers, SSNs, credit cards, IP addresses, URLs, API keys — swaps
them for stable placeholders like `[EMAIL_1]`, and gives you a mapping to put the
originals back afterward. Send the *redacted* text to a third-party AI API, get a
response that still references the placeholders, and restore it locally.

- Pure Python standard library. No dependencies.
- Python 3.8+.
- Reversible round-trip by design (`restore(redact(t).text) == t`).
- Luhn-validated credit-card detection to cut down false positives.
- Pluggable custom detectors with optional validators.

## Install

```bash
pip install ragkit-redactor
```

Local development (from `redactor/`):

```bash
pip install -e .
```

## Quick Start

```python
import redactor

text = (
    "Hi, this is Alice. Reach me at alice@example.com or +1 (555) 123-4567. "
    "My card is 4111 1111 1111 1111 if you need to charge the deposit."
)

result = redactor.redact(text)

print(result.text)
# Hi, this is Alice. Reach me at [EMAIL_1] or [PHONE_1].
# My card is [CREDIT_CARD_1] if you need to charge the deposit.

print(result.mapping)
# {'[EMAIL_1]': 'alice@example.com',
#  '[PHONE_1]': '+1 (555) 123-4567',
#  '[CREDIT_CARD_1]': '4111 1111 1111 1111'}

# --- send result.text to your LLM of choice ---
llm_reply = "Sure — I'll confirm the deposit and email [EMAIL_1] the receipt."

# Un-mask the model's output locally:
print(result.restore(llm_reply))
# Sure — I'll confirm the deposit and email alice@example.com the receipt.
```

The redacted text is safe(r) to send to a third-party API; the sensitive values
never leave your process.

## API Reference

### `redactor.redact(text, **kwargs) -> RedactionResult`

Convenience wrapper: builds a `Redactor(**kwargs)` and redacts `text` in one call.

### `redactor.luhn_check(number_str) -> bool`

Runs the Luhn checksum over the digits in `number_str` (ignoring spaces/dashes).
Used internally to validate credit-card candidates; exposed for your own use.

### `class Redactor(detectors=None, placeholder_template="[{type}_{n}]", mask_char=None)`

- **`detectors`** — list of detector names to enable. Defaults to all built-ins.
  Pass `[]` to start empty and add your own.
- **`placeholder_template`** — format string with `{type}` and `{n}` fields.
  Default `"[{type}_{n}]"` produces `[EMAIL_1]`, `[PHONE_2]`, etc.
- **`mask_char`** — if set (e.g. `"*"`), matches are replaced by that character
  repeated to the length of the match. **This mode is one-way** (see below).

#### `add_detector(name, pattern, validator=None)`

Register a custom detector. `pattern` is a regex string or compiled pattern.
`validator(str) -> bool` optionally filters raw matches (return `False` to reject).
Custom detectors take priority over built-ins for overlap resolution.

```python
r = redactor.Redactor()
r.add_detector("EMP_ID", r"EMP-\d{4}", validator=lambda s: s != "EMP-0000")
```

#### `detect(text) -> List[Match]`

Returns all matches across enabled detectors, with overlaps resolved and sorted
by start offset. Overlap resolution is deterministic: earlier start wins; for the
same start the longer span wins; remaining ties go to detector priority
(registration order, custom detectors first).

#### `redact(text) -> RedactionResult`

Detects, then substitutes placeholders (or mask characters). Identical values get
the **same** placeholder (dedupe). Output is built by walking matches
left-to-right, so offsets never corrupt.

#### `restore(text, mapping=None) -> str`

Replaces placeholders in `text` with their originals using `mapping` (required —
a bare `Redactor` keeps no state). Longer placeholders are substituted first so
`[X_1]` can't be clobbered by `[X_11]`.

### `class Match`

Dataclass: `type` (detector name), `value` (original text), `start`, `end`
(offsets in the source), `placeholder` (assigned during `redact`, else `None`).

### `class RedactionResult`

- **`.text`** — the redacted string.
- **`.matches`** — list of `Match` with placeholders assigned.
- **`.mapping`** — `{placeholder: original_value}` (empty in `mask_char` mode).
- **`.restore(model_output)`** — un-mask placeholders in an LLM response using
  this result's mapping. This is the reversible round-trip.

## Built-in Detectors

| Name | What it matches |
|------|-----------------|
| `EMAIL` | Email addresses |
| `PHONE` | US-style and international-ish numbers (`+country`, separators, parentheses) |
| `SSN` | US Social Security numbers (`###-##-####`) |
| `CREDIT_CARD` | 13–16 digit sequences (spaces/dashes allowed) **that pass the Luhn check** |
| `IP_ADDRESS` | IPv4 addresses |
| `IPV6` | IPv6 addresses (basic) |
| `URL` | `http`/`https` URLs |
| `API_KEY` | `sk-...` tokens and long hex secrets (32+ chars) |
| `AWS_ACCESS_KEY` | `AKIA` + 16 uppercase alphanumerics |

## False positives & Luhn validation

Regex PII detection is a balance between catching real data and not flagging every
number in your prose. A few pragmatic choices:

- **Credit cards** are only reported when the digits pass `luhn_check`, which
  discards the vast majority of random 13–16 digit strings.
- **API keys / secrets** are kept deliberately narrow (`sk-` tokens and long hex
  blobs) rather than "anything that looks random", to avoid swallowing ordinary
  words and IDs.
- **Overlaps** (e.g. an IP embedded inside a URL) collapse to a single match, so
  you never get doubled or garbled output.

You will still see occasional misses and occasional false hits. Tune with the
`detectors` list and `add_detector`.

## Reversible placeholders vs. `mask_char`

There are two modes:

- **Placeholder mode (default, reversible).** Values become `[TYPE_n]` tokens and
  a `mapping` is returned. You can `restore()` both your text and the LLM's reply.
  `restore(redact(t).text, mapping) == t`.
- **`mask_char` mode (one-way).** Set `mask_char="*"` and each match becomes `****`
  of equal length. No mapping is produced and **restoration is impossible** — use
  this only when you want to scrub data for display/logging, not round-trip it.

```python
r = redactor.Redactor(mask_char="*")
print(r.redact("SSN 123-45-6789").text)   # SSN ***********
```

## A note on compliance

This library is **best-effort**. Regex-based detection cannot catch every form of
sensitive data, and matching data is not the same as understanding it. `redactor`
is a practical safety net, **not** a guarantee of GDPR / HIPAA / PCI-DSS
compliance or any other regulatory requirement. Review the redacted output before
relying on it, and don't treat it as a substitute for proper data-handling
controls.

## License

MIT.
