Metadata-Version: 2.5
Name: rahasya
Version: 0.0.1
Summary: thirty-four kinds of personal identifier, found by arithmetic: checksummed patterns read in context, no model and no network
Project-URL: Repository, https://github.com/vedicreader/rahasya
Project-URL: Documentation, https://vedicreader.github.io/rahasya/
Author-email: Karthik <karthik.rajgopal@hotmail.com>
License: Apache-2.0
License-File: LICENSE
Keywords: checksum,dlp,gdpr,nbdev,ner,pii,privacy,redaction
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: Text Processing
Requires-Python: >=3.12
Requires-Dist: fastcore>=1.12.31
Description-Content-Type: text/markdown

# rahasya


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

Thirty-four kinds of personal identifier, each a pattern with a checksum, read in context.
**1.000 precision at recall 1.000** on a held-out corpus; 0.996 on the corpus they were tuned
against. No model, no network, no gigabyte on disk.

## Install

``` sh
uv add rahasya
```

## The one call

``` python
r = pii_report('Invoice for ada@example.com. Card 4111 1111 1111 1111. '
                'KEY=sk-abcdefghijklmnopqrstuvwxyz123456')
r.has_pii, r.identifying
```

`has_pii` counts only `IDENTIFYING` kinds. An IP address is reported and does not tip the
gate.

``` python
redact('Card 4111 1111 1111 1111 charged to ada@example.com')
```

## Digits are read in context

A number that fails its checksum is not the thing the checksum protects, and a number introduced
by a word that makes it a reference is not an identifier at all.

``` python
[pii_report(t).has_pii for t in
 ('Order 4111 1111 1111 1112 shipped',   # fails Luhn
  'Conforms to EN 60601-1',              # a standard, not a ZIP
  'Card 4111 1111 1111 1111')]           # this one is a card
```

`DESIGNATOR` holds the words that do it: `ISO`, `RFC`, `invoice`, `page`, `commit`, `order`
and thirty more. It is matched against the 48 characters to the left of a candidate.

## Nineteen regional identifiers, each with its own checksum

Aadhaar, PAN and GSTIN; Australian TFN, ABN and Medicare; Singapore NRIC; Thai national ID; Dutch
BSN; French NIR; Spanish DNI and NIE; Italian codice fiscale; Polish PESEL; Swedish personnummer;
Norwegian fødselsnummer; German Steuer-IdNr; UK NINO; IMEI.

``` python
import random
a = gen_aadhaar(random.Random(0))            # a valid one, for a fixture
a, aadhaar_ok(a), pesel_ok('44051401359'), nino_ok('AB123456C')
```

## Names are opt-in

`ner=True` turns on an honorific-anchored pass. `Dr Charles Babbage` matches; a bare
`Ada Lovelace` does not. `scanned_ner` says whether anything looked, which keeps “none found”
apart from “not looked for”.

``` python
(pii_report('Dr Charles Babbage signed it.').has_pii,
 pii_report('Dr Charles Babbage signed it.', ner=True).identifying,
 redact('Dr Charles Babbage signed it.', ner=True))
```

## What is here

| name | what it does |
|----|----|
| `pii_report` | spans found, and whether they tip `has_pii` |
| `pii_spans` | `(start, end, kind, value)`, de-overlapped longest-first |
| `redact` | mask matched spans; `[EMAIL]`, `[CARD]`, or a mask you name |
| `redact_obj` | `redact` over the strings inside a nested dict or list |
| `person_names` / `person_spans` | the honorific-anchored name pass |
| `PATTERNS` | kind to `(pattern, validator)`; the whole detector as data |
| `IDENTIFYING` | which kinds tip the gate |

`luhn`, `aadhaar_ok`, `pesel_ok` and the rest are exported too, because a checksum is useful on
its own.

## Why no model

Two learned detectors were measured against these patterns and neither is shipped. Both lose on
precision *and* recall, and are 200 times slower. A DeBERTa-v3 ONNX classifier earns its gigabyte
only on names no honorific introduces (2/8 to 5/8 of them). A 350M tflite encoder finds no names
at all.

## Development

The notebooks in `nbs/` are the source; the modules are generated.

``` sh
pip install -e .
nbdev_prepare
```
