Metadata-Version: 2.5
Name: llmveil
Version: 0.1.0
Summary: Reversible PII redaction for LLM pipelines. Local, deterministic, no network calls.
Project-URL: Homepage, https://github.com/larsgrosscom/llmveil
Project-URL: Repository, https://github.com/larsgrosscom/llmveil
Project-URL: Issues, https://github.com/larsgrosscom/llmveil/issues
Project-URL: Changelog, https://github.com/larsgrosscom/llmveil/blob/main/CHANGELOG.md
Author: Lars Gross
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: anonymization,data-protection,llm,pii,privacy,prompt,pseudonymization,redaction
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: Text Processing :: Filters
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: tomli>=2.0; python_version < '3.11'
Provides-Extra: all
Requires-Dist: cryptography>=42.0; extra == 'all'
Requires-Dist: spacy>=3.7; extra == 'all'
Provides-Extra: crypto
Requires-Dist: cryptography>=42.0; extra == 'crypto'
Provides-Extra: dev
Requires-Dist: cryptography>=42.0; extra == 'dev'
Requires-Dist: hypothesis>=6.100; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff<0.17,>=0.16; extra == 'dev'
Provides-Extra: ner
Requires-Dist: spacy>=3.7; extra == 'ner'
Description-Content-Type: text/markdown

# llmveil

Reversible PII redaction for LLM pipelines. Masks personal data before text
reaches a model, and puts the original values back in the answer.

Local, deterministic, and it never touches the network.

```bash
pip install llmveil
```

```python
from llmveil import redact, restore

result = redact("Email anna@example.de about invoice 4711.", locales=["de_DE"])
print(result.masked)                      # Email [EMAIL_1] about invoice 4711.
answer = call_your_model(result.masked)   # the model never sees the address
print(restore(answer, result.mapping))    # ...and you get it back
```

## Read this first

Detection is best effort and incomplete. **This library reduces exposure. It
does not guarantee that no personal data reaches the model, and it is not a
compliance control.** Measured coverage, including what it misses, is in the
table below. What it does and does not protect against is in
[docs/threat-model.md](docs/threat-model.md).

## What it does

**Validates instead of guessing.** Anything with a check digit gets it verified:
Luhn, IBAN mod-97 with per-country lengths, NHS mod-11, the German tax ID and
its repeated-digit rule, the USt-IdNr, the Rentenversicherungsnummer weighted
sum, ABA, Codice Fiscale, DNI. Where a checksum exists, a pattern without a
validator is a false-positive generator.

**Uses context, weighted by distance.** `Bestellnummer 47036892816` is a
checksum-valid German tax ID by shape. The word in front of it says otherwise,
and the nearest keyword wins, so a label two clauses away cannot override the
one sitting right next to the number.

**Restores tolerantly.** Models return `**[PERSON_1]**`, `<person_1>`,
`[ PERSON_1 ]`, or the same placeholder five times. All of that restores. A
placeholder the model *invented* is left exactly where it is and reported,
because substituting a value there would be fabricating data.

**Treats the mapping as a secret.** It is never in a `repr`, never in an
exception message, never in the stats, never in a scan report. `Span` objects
carry offsets and labels only, so they are safe to log.

**Never uses the network.** Enforced by a test that blocks `socket` and then
exercises every code path, including a check that the block itself works.

## Measured coverage

Run `python evaluation/run_eval.py` to reproduce. The corpus is 33 labelled
cases, 13 of them hard negatives.

| Entity | Precision | Recall | F1 | TP | FP | FN |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| `ADDRESS` | 1.00 | 1.00 | 1.00 | 2 | 0 | 0 |
| `API_KEY` | 1.00 | 1.00 | 1.00 | 2 | 0 | 0 |
| `CREDIT_CARD` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `DE_HANDELSREGISTER` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `DE_POSTAL_CODE` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `DE_SOZIALVERSICHERUNG` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `DE_STEUER_ID` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `DE_USTIDNR` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `EMAIL` | 1.00 | 1.00 | 1.00 | 2 | 0 | 0 |
| `GB_ACCOUNT_NUMBER` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `GB_NHS_NUMBER` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `GB_NINO` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `GB_POSTCODE` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `GB_SORT_CODE` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `IBAN` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| **`PERSON`** | **1.00** | **0.00** | **0.00** | **0** | **0** | **2** |
| `PHONE` | 1.00 | 1.00 | 1.00 | 2 | 0 | 0 |
| `US_EIN` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `US_SSN` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| `US_ZIP` | 1.00 | 1.00 | 1.00 | 1 | 0 | 0 |
| **overall** | **1.00** | **0.92** | **0.96** | **23** | **0** | **2** |

### The gaps, which are the useful part

**`PERSON` recall is zero without the NER extra.** Names have no fixed shape,
so nothing in the deterministic layer can find them. If names are your concern,
`pip install llmveil[ner]` and set `use_ner=True`. Without it, plan another
control for names.

**Only three locales are complete.** en_US, de_DE and en_GB. fr_FR, es_ES,
it_IT, nl_NL, ch_CH and at_AT are scaffolded and raise a clear error rather
than silently detecting nothing.

**Unvalidated identifiers depend on context.** Driver's licence numbers, US
licence plates, passport numbers, UK account numbers and UTRs have no checksum
and shapes that collide with ordinary strings. They ship at low confidence and
only survive when a keyword supports them. Expect misses when the surrounding
text gives no hint.

**This corpus is small and written by the author.** It exercises the hard paths
deliberately. Numbers on your own data will be lower. Run `llmveil scan` over a
sample of your corpus before trusting any of this.

## Performance

```
10 240 chars   min 17.7 ms   median 24.1 ms   128 detections
```

Python 3.10 on Windows, the slowest supported configuration. **This misses the
10 ms target that was set for it, by a wide margin.** The time goes into 38
independent regex passes over the same text. The fix would be to combine them
into a single alternation, which is a rewrite with real correctness risk and
has not been done.

The median is load dependent and not worth quoting precisely: the same
unchanged code measured anywhere between 13 and 30 ms on the same machine
depending on what else was running. The minimum is the stable figure, because
noise only ever makes a run slower, never faster, and it is what
`benchmarks/run_benchmark.py --baseline` gates on. Measure on your own hardware
before planning around any of these numbers.

## The API

```python
from llmveil import Redactor, redact, restore, restore_report, scan

result = redact(text, locales=["de_DE"])
result.masked      # str
result.mapping     # Mapping: placeholder -> original. This is the secret.
result.spans       # list[Span]: offsets, labels, confidence. No values.
result.stats       # counts and timing. No values.

text, report = restore_report(model_output, result.mapping)
report.restored, report.missing, report.unknown, report.repeated
```

Build a `Redactor` once and share it. It is immutable after construction and
safe to use from several threads; `add_pattern` returns a new instance rather
than mutating a shared one.

### Wrapping a model call

```python
from llmveil.adapters import protect, protect_runnable

# any object with .invoke(str), or any callable taking a string
chain = protect_runnable(prompt | llm | parser, locales=["de_DE"])
answer = chain.invoke("Schreib an anna@example.de")

@protect(locales=["de_DE"])
def ask(prompt: str) -> str:
    return call_the_model(prompt)
```

Both clear the mapping when the call returns, so it never outlives the
request that produced it.

### Multi-turn conversations

Pass the mapping forward. A seed alone is not enough, and this is documented
rather than left to be discovered later:

```python
first = redact(turn_one, locales=["de_DE"])
second = redact(turn_two, locales=["de_DE"], mapping=first.mapping)
# anyone seen in turn one keeps the same placeholder in turn two
```

### Options

| Option | Default | Effect |
| --- | --- | --- |
| `locales` | `("en_US",)` | Locale packs. Universal detectors always run. |
| `placeholder_style` | `"bracket"` | `bracket`, `angle`, `token`, `brace`, `surrogate` |
| `min_confidence` | `0.4` | Global floor; `thresholds` overrides per entity |
| `allowlist`, `allowlist_patterns` | `()` | Never redact these |
| `denylist` | `()` | Always redact these |
| `entities`, `exclude_entities` | | Restrict what is detected |
| `use_context` | `True` | The distance-weighted keyword layer |
| `use_entropy` | `True` | Generic high-entropy secret detection |
| `use_ner` | `False` | Needs `llmveil[ner]` |
| `explain` | `False` | Record why each span matched |
| `strict` | `False` | Raise instead of reporting |

Config comes from code, a dict, a TOML `[llmveil]` table, or `LLMVEIL_*`
environment variables. A misspelled environment variable is an error, not a
silent default: a security setting must not fail open.

### Placeholder styles

`bracket` is the default and the safest. It survives tokenisation as a stable
token sequence the model has no reason to alter, and it restores exactly.

`surrogate` generates realistic fakes instead: a German name for a German name,
an IBAN with a genuinely valid checksum, a Luhn-valid card in a published test
range. Use it when a model behaves badly on bracket tokens. **It restores less
reliably**, because a fake name is a word and models inflect words: "Müller"
comes back as "Müllers Anfrage" and no longer matches. That trade is why
bracket is the default.

## Command line

```bash
llmveil redact notes.txt -l de_DE --mapping map.json -o masked.txt
llmveil restore masked.txt --mapping map.json
llmveil scan ./docs --report              # find PII, change nothing, print no values
llmveil scan ./docs --fail-on-findings    # as a CI gate
llmveil locales
```

`scan` is worth using on its own, before any model is involved. It never prints
a detected value, only offsets, labels and counts.

## Installation

```bash
pip install llmveil              # deterministic and context layers
pip install llmveil[ner]         # adds names, organisations, locations
pip install llmveil[crypto]      # encrypted mapping serialisation
pip install llmveil[all]
```

There is no framework extra. The model wrapper is duck-typed and imports
nothing, so it works with LangChain, LlamaIndex or a plain function without
any of them being installed.

Python 3.10+. The core has no dependencies at all, except `tomli` on 3.10,
which has no `tomllib` in the standard library. All dependency licences are
Apache, MIT or BSD.

The NER extra needs a model, installed once by you and never downloaded by this
library:

```bash
pip install llmveil[ner]
python -m spacy download de_core_news_sm
```

## Development

```bash
pip install -e ".[dev]"
pytest                              # 354 tests
ruff check . && ruff format --check . && mypy
python evaluation/run_eval.py       # regenerates the table above
python benchmarks/run_benchmark.py
```

The numbers in this README are generated, not typed. If the table and
`evaluation/run_eval.py` disagree, the table is wrong.

## Documentation

- [Threat model](docs/threat-model.md), what this protects against and what it does not
- [Locales](docs/locales.md), coverage per country and how to add one
- [Custom patterns](docs/custom-patterns.md)
- [Tuning precision](docs/precision-tuning.md)

## Author

Built by Lars Gross. More of my work at [larsgross.com](https://larsgross.com).

## License

Apache-2.0. See [LICENSE](LICENSE).
