Metadata-Version: 2.5
Name: credgate
Version: 0.3.0
Summary: Governance layer for LLM requests with Indian PII detection
Project-URL: Homepage, https://github.com/dwarpal/credgate
Project-URL: Issues, https://github.com/dwarpal/credgate/issues
Project-URL: Changelog, https://github.com/dwarpal/credgate/blob/main/pkg/python/CHANGELOG.md
Author-email: Bharat Sikka <bharat8sikka@gmail.com>, Aishwarya Bais <aishwarya.bais00@gmail.com>
Maintainer-email: Bharat Sikka <bharat8sikka@gmail.com>, Aishwarya Bais <aishwarya.bais00@gmail.com>
License-Expression: Apache-2.0
License-File: AUTHORS
License-File: LICENSE
License-File: NOTICE
Keywords: aadhaar,compliance,dpdp,india,llm,pii,privacy
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Security
Classifier: Topic :: Text Processing
Requires-Python: >=3.11
Requires-Dist: pyyaml>=6.0
Requires-Dist: regex>=2023.0
Provides-Extra: test
Requires-Dist: pytest>=7; extra == 'test'
Description-Content-Type: text/markdown

# credgate

Detects personal data before it leaves the machine, with Indian identifiers
built in: Aadhaar (with Verhoeff checksum), Aadhaar Virtual ID, PAN, GSTIN,
voter ID, passport, driving licence, UPI IDs, IFSC, bank accounts, payment
cards, +91 mobile numbers, ABHA, and names and addresses in Latin and Indic
scripts.

The detection rules live in a shared, language-neutral spec, so this package
and the R package `credgate` find exactly the same matches. Part of the
[CredGate](https://github.com/dwarpal/credgate) project; early development.

## Use it with local LLMs (Ollama, vLLM)

Redact before the prompt reaches the model; the model only ever sees
placeholders such as `[PAN]` and `[PERSON_NAME]`.

```python
import credgate, ollama

note = "Customer name: Priya Nair. PAN ABCPN1234K, mobile +91 98765 43210, wants the EMI date moved."
print(credgate.preview(note, min_confidence=0.5))   # what will be masked, and why
safe = credgate.redact(note, min_confidence=0.5)    # "Customer name: [PERSON_NAME]. PAN [PAN], mobile [MOBILE_IN], ..."
reply = ollama.chat(model="gemma3:1b", messages=[{"role": "user", "content": f"Summarise: {safe}"}])
```

With **vLLM** (or Ollama's `/v1`, LM Studio, any OpenAI-compatible server):

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
client.chat.completions.create(model="<served model>",
                               messages=[{"role": "user", "content": credgate.redact(note, min_confidence=0.5)}])
```

From the shell: `credgate redact notes.txt | ollama run gemma3:1b "Summarise:"`.
More runnable examples (ticket classification, RAG ingestion) are in the
[examples folder](https://github.com/dwarpal/credgate/tree/main/examples).

> **Check the preview.** Names are found next to an honorific, relation or
> field label (`Shri`, `S/o`, `Name:`, `नाम:`); a name in running text
> without such a cue ("Customer Priya Nair called") is not detected yet.
> Redaction is irreversible: the model's answer cannot mention the real
> values (reversible tokenisation and an automatic proxy are on the roadmap).

## How it compares

| On 700 held-out synthetic documents | Precision | Recall | **F1** | Time |
|---|---:|---:|---:|---:|
| **credgate** | 0.985 | 0.916 | **0.949** | 0.3 s |
| Presidio, India-configured | 0.600 | 0.748 | 0.666 | 12.1 s |
| Presidio, default | 0.555 | 0.472 | 0.510 | 15.6 s |

<sub>**About the data:** the 700 documents are **synthetic** - KYC forms, bank
messages, call-centre notes, Hinglish chats, Hindi text and hospital
summaries generated by
[`benchmark/generate_corpus.py`](https://github.com/dwarpal/credgate/blob/main/benchmark/generate_corpus.py)
and included in the repository under
[`benchmark/corpus/`](https://github.com/dwarpal/credgate/tree/main/benchmark/corpus).
No real personal data was used.
Types both tools cover, exact span matching, thresholds tuned on a
separate dev split for every system; time is for 1,400 documents. Synthetic
data generated by us, so read with that in mind.</sub>

credgate is built for Indian identifiers and Indian-language context, and
that is what this benchmark measures. Presidio is broader and more mature:
it detects many types credgate does not (international IDs, IBAN, IP
addresses, URLs, dates), finds names without an honorific or label, offers
reversible encryption, redacts images, and wins here on voter IDs without
context. Method, per-type results, where Presidio is ahead, and how to
reproduce: [benchmark/README.md](https://github.com/dwarpal/credgate/blob/main/benchmark/README.md).
The comparison is independent and not affiliated with Microsoft.

## Installation

Python 3.11 or later; depends only on `regex` and `PyYAML`.

```sh
pip install credgate
```

The development version: `pip install "git+https://github.com/dwarpal/credgate#subdirectory=pkg/python"`.

With Nix: `nix build github:dwarpal/credgate#credgate`.

## Usage

```python
import credgate

spans = credgate.detect_pii("Name: Priya Nair; PAN: ABCPN1234K; UPI: priya.n@oksbi")
for s in spans:
    print(s)
# Span(doc=0, start=6, end=16, type='person_name', value='XXXXX Nair', confidence=0.8)
# Span(doc=0, start=23, end=33, type='pan', value='XXXXXX234K', confidence=0.97)
# Span(doc=0, start=40, end=53, type='upi_id', value='XXXXX.X@Xksbi', confidence=0.99)

spans[1].value    # 'ABCPN1234K' - printing masks values, the attribute does not
spans[1].reasons  # "matched PAN pattern; context 'PAN'; holder type: individual"
```

`detect_pii()` takes a string or a list of strings and returns `Span`
objects with the document index, 0-based end-exclusive positions, type, data
class, value, canonical form, confidence and reasons. `span.to_dict()` gives
a JSON-ready dict with the value masked (`reveal=True` to include it).

Redact identifiers, or preview what would be redacted and why:

```python
credgate.redact("Name: Priya Nair; PAN: ABCPN1234K")
# 'Name: [PERSON_NAME]; PAN: [PAN]'

print(credgate.preview("Name: Priya Nair; PAN: ABCPN1234K; order no 878245671885"))
# 3 items would be redacted:
#   Name: [PERSON_NAME]; PAN: [PAN]; order no [AADHAAR]
#
#   [PERSON_NAME]  XXXXX Nair    0.80  matched Person name pattern
#   [PAN]          XXXXXX234K    0.97  matched PAN pattern; context 'PAN'; ...
#   [AADHAAR]      XXXXXXXX1885  0.42  ...; negative context 'order'
```

Scan a table (a pandas DataFrame, or a dict of column -> values) to see
which columns hold PII; column names count as context:

```python
report = credgate.scan_pii({
    "cust_pan": ["ABCPE1234F", "AAACR5055K", "XYZPQ9876R"],
    "notes": ["Called, Aadhaar 542229371182", "no issues", "mobile 9876543210"],
})
print(report)
# Scanned 2 column(s); PII found in 2: cust_pan, notes
#   cust_pan  pan              100% of 3 (3 whole value)  confidence 0.97  e.g. XXXXXX234F
#   notes     aadhaar           33% of 3 (0 whole value)  confidence 0.97  e.g. XXXXXXXX1182
#   notes     mobile_in         33% of 3 (0 whole value)  confidence 0.93  e.g. XXXXXX3210
pandas.DataFrame(report.to_records())  # as a table
```

Context makes generic patterns trustworthy; pass a column name or similar:

```python
credgate.detect_pii("ABC1234567")                      # [] - could be anything
credgate.detect_pii("ABC1234567", context="voter_id")  # voter ID, confidence 0.97
```

Checksums, synthetic test data and your own detectors:

```python
credgate.verhoeff_valid("542229371182")
credgate.synth_indian_pii(5, seed=1)   # synthetic records; never use as anonymised data

credgate.register_detector(credgate.Detector(
    id="employee_id", data_class="internal", label="Employee ID",
    patterns=credgate.bounded("EMP[0-9]{6}"),
))
```

## Command line

```sh
credgate detect notes.txt                      # masked report
credgate detect --lines --json customers.csv   # one JSON object per match
git diff --cached | credgate detect --fail     # non-zero exit if PII is found
credgate redact notes.txt > notes.redacted.txt  # replace PII with [PAN] etc.
credgate preview notes.txt                     # what would be redacted, and why
credgate scan customers.csv                    # which columns hold PII
credgate detectors                             # list detectors
```

Values are masked unless you pass `--reveal`.

## Limits

Names and addresses are found only next to a cue: an honorific (Shri, Smt,
श्रीमती), a relation (S/o, D/o), a field label (`Name:`, `पता:`) or a
house/village keyword followed by a PIN code. A bare 12-digit number may pass
the Aadhaar checksum by chance and is reported with lower confidence.

## Licence

Apache License 2.0.
