Metadata-Version: 2.5
Name: credgate
Version: 0.3.1
Summary: Governance layer for LLM requests with Indian PII detection
Project-URL: Homepage, https://github.com/dwarpal/credgate
Project-URL: Issues, https://github.com/dwarpal/credgate/issues
Project-URL: Changelog, https://github.com/dwarpal/credgate/blob/main/pkg/python/CHANGELOG.md
Author-email: Bharat Sikka <bharat8sikka@gmail.com>, Aishwarya Bais <aishwarya.bais00@gmail.com>
Maintainer-email: Bharat Sikka <bharat8sikka@gmail.com>, Aishwarya Bais <aishwarya.bais00@gmail.com>
License-Expression: Apache-2.0
License-File: AUTHORS
License-File: LICENSE
License-File: NOTICE
Keywords: aadhaar,compliance,dpdp,india,llm,pii,privacy
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Security
Classifier: Topic :: Text Processing
Requires-Python: >=3.11
Requires-Dist: pyyaml>=6.0
Requires-Dist: regex>=2023.0
Provides-Extra: test
Requires-Dist: pytest>=7; extra == 'test'
Description-Content-Type: text/markdown

# credgate

Detects personal data before it leaves the machine, with Indian identifiers
built in: Aadhaar (with Verhoeff checksum), Aadhaar Virtual ID, PAN, GSTIN,
voter ID, passport, driving licence, UPI IDs, IFSC, bank accounts, payment
cards, +91 mobile numbers, ABHA, and names and addresses in Latin and Indic
scripts.

The detection rules live in a shared, language-neutral spec, so this package
and the R package `credgate` find exactly the same matches. Part of the
[CredGate](https://github.com/dwarpal/credgate) project; early development.

## Use it with local LLMs (Ollama, vLLM)

Redact before the prompt reaches the model; the model only ever sees
placeholders such as `[PAN]` and `[PERSON_NAME]`.

```python
import credgate, ollama

note = "Customer name: Priya Nair. PAN ABCPN1234K, mobile +91 98765 43210, wants the EMI date moved."
print(credgate.preview(note, min_confidence=0.5))   # what will be masked, and why
safe = credgate.redact(note, min_confidence=0.5)    # "Customer name: [PERSON_NAME]. PAN [PAN], mobile [MOBILE_IN], ..."
reply = ollama.chat(model="gemma3:1b", messages=[{"role": "user", "content": f"Summarise: {safe}"}])
```

With **vLLM** (or Ollama's `/v1`, LM Studio, any OpenAI-compatible server):

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
client.chat.completions.create(model="<served model>",
                               messages=[{"role": "user", "content": credgate.redact(note, min_confidence=0.5)}])
```

From the shell: `credgate redact notes.txt | ollama run gemma3:1b "Summarise:"`.
More runnable examples (ticket classification, RAG ingestion) are in the
[examples folder](https://github.com/dwarpal/credgate/tree/main/examples).

> **Check the preview.** Names are found next to an honorific, relation or
> field label (`Shri`, `S/o`, `Name:`, `नाम:`); a name in running text
> without such a cue ("Customer Priya Nair called") is not detected yet.
> Redaction is irreversible: the model's answer cannot mention the real
> values (reversible tokenisation and an automatic proxy are on the roadmap).

## How it compares

| On 700 held-out synthetic documents | Precision | Recall | **F1** | Time |
|---|---:|---:|---:|---:|
| **credgate** | 0.985 | 0.916 | **0.949** | 0.3 s |
| Presidio, India-configured | 0.600 | 0.748 | 0.666 | 12.1 s |
| Presidio, default | 0.555 | 0.472 | 0.510 | 15.6 s |

<sub>**About the data:** the 700 documents are **synthetic** - KYC forms, bank
messages, call-centre notes, Hinglish chats, Hindi text and hospital
summaries generated by
[`benchmark/generate_corpus.py`](https://github.com/dwarpal/credgate/blob/main/benchmark/generate_corpus.py)
and included in the repository under
[`benchmark/corpus/`](https://github.com/dwarpal/credgate/tree/main/benchmark/corpus).
No real personal data was used.
Types both tools cover, exact span matching, thresholds tuned on a
separate dev split for every system; time is for 1,400 documents. Synthetic
data generated by us, so read with that in mind.</sub>

**Where Presidio does better.** credgate is built for Indian identifiers
and Indian-language context, and that is what this benchmark measures.
Presidio is broader and more mature:

- **Voter IDs without context**: Presidio scores 1.000 here, credgate 0.677
  (credgate needs a nearby keyword such as "voter" or "EPIC").
- **Many more entity types**: international IDs (US, UK, EU, ...), IBAN, IP
  and MAC addresses, URLs, crypto wallets, dates, nationality and religion -
  credgate detects none of these.
- **Names in running text**: Presidio's NER finds names without an honorific
  or field label; credgate does not yet.
- **Anonymisation**: Presidio can mask, hash or reversibly encrypt;
  credgate only redacts irreversibly so far.
- **Beyond text**: image (OCR) and structured-data redaction, a REST service
  and Docker images.
- **Maturity and extensibility**: years in production, many languages and
  NLP back-ends, custom recognisers; credgate is pre-alpha.
- **The corpus favours credgate**: we generated it; on general English text
  Presidio may well do better.

Full method, per-type results and how to reproduce:
[benchmark/README.md](https://github.com/dwarpal/credgate/blob/main/benchmark/README.md).
The comparison is independent and not affiliated with Microsoft.

## Installation

Python 3.11 or later; depends only on `regex` and `PyYAML`.

```sh
pip install credgate
```

The development version: `pip install "git+https://github.com/dwarpal/credgate#subdirectory=pkg/python"`.

With Nix: `nix build github:dwarpal/credgate#credgate`.

## Usage

```python
import credgate

spans = credgate.detect_pii("Name: Priya Nair; PAN: ABCPN1234K; UPI: priya.n@oksbi")
for s in spans:
    print(s)
# Span(doc=0, start=6, end=16, type='person_name', value='XXXXX Nair', confidence=0.8)
# Span(doc=0, start=23, end=33, type='pan', value='XXXXXX234K', confidence=0.97)
# Span(doc=0, start=40, end=53, type='upi_id', value='XXXXX.X@Xksbi', confidence=0.99)

spans[1].value    # 'ABCPN1234K' - printing masks values, the attribute does not
spans[1].reasons  # "matched PAN pattern; context 'PAN'; holder type: individual"
```

`detect_pii()` takes a string or a list of strings and returns `Span`
objects with the document index, 0-based end-exclusive positions, type, data
class, value, canonical form, confidence and reasons. `span.to_dict()` gives
a JSON-ready dict with the value masked (`reveal=True` to include it).

Redact identifiers, or preview what would be redacted and why:

```python
credgate.redact("Name: Priya Nair; PAN: ABCPN1234K")
# 'Name: [PERSON_NAME]; PAN: [PAN]'

print(credgate.preview("Name: Priya Nair; PAN: ABCPN1234K; order no 878245671885"))
# 3 items would be redacted:
#   Name: [PERSON_NAME]; PAN: [PAN]; order no [AADHAAR]
#
#   [PERSON_NAME]  XXXXX Nair    0.80  matched Person name pattern
#   [PAN]          XXXXXX234K    0.97  matched PAN pattern; context 'PAN'; ...
#   [AADHAAR]      XXXXXXXX1885  0.42  ...; negative context 'order'
```

Scan a table (a pandas DataFrame, or a dict of column -> values) to see
which columns hold PII; column names count as context:

```python
report = credgate.scan_pii({
    "cust_pan": ["ABCPE1234F", "AAACR5055K", "XYZPQ9876R"],
    "notes": ["Called, Aadhaar 542229371182", "no issues", "mobile 9876543210"],
})
print(report)
# Scanned 2 column(s); PII found in 2: cust_pan, notes
#   cust_pan  pan              100% of 3 (3 whole value)  confidence 0.97  e.g. XXXXXX234F
#   notes     aadhaar           33% of 3 (0 whole value)  confidence 0.97  e.g. XXXXXXXX1182
#   notes     mobile_in         33% of 3 (0 whole value)  confidence 0.93  e.g. XXXXXX3210
pandas.DataFrame(report.to_records())  # as a table
```

Context makes generic patterns trustworthy; pass a column name or similar:

```python
credgate.detect_pii("ABC1234567")                      # [] - could be anything
credgate.detect_pii("ABC1234567", context="voter_id")  # voter ID, confidence 0.97
```

Checksums, synthetic test data and your own detectors:

```python
credgate.verhoeff_valid("542229371182")
credgate.synth_indian_pii(5, seed=1)   # synthetic records; never use as anonymised data

credgate.register_detector(credgate.Detector(
    id="employee_id", data_class="internal", label="Employee ID",
    patterns=credgate.bounded("EMP[0-9]{6}"),
))
```

## Command line

```sh
credgate detect notes.txt                      # masked report
credgate detect --lines --json customers.csv   # one JSON object per match
git diff --cached | credgate detect --fail     # non-zero exit if PII is found
credgate redact notes.txt > notes.redacted.txt  # replace PII with [PAN] etc.
credgate preview notes.txt                     # what would be redacted, and why
credgate scan customers.csv                    # which columns hold PII
credgate detectors                             # list detectors
```

Values are masked unless you pass `--reveal`.

## Limits

Names and addresses are found only next to a cue: an honorific (Shri, Smt,
श्रीमती), a relation (S/o, D/o), a field label (`Name:`, `पता:`) or a
house/village keyword followed by a PIN code. A bare 12-digit number may pass
the Aadhaar checksum by chance and is reported with lower confidence.

## Licence

Apache License 2.0.
