Metadata-Version: 2.4
Name: langchain-sct
Version: 0.1.0
Summary: Reversible PII pseudonymization + token compression for LangChain 1.0 — a keyed, DSGVO/BaFin-grade superset of the built-in PIIMiddleware.
Project-URL: Homepage, https://sct.simosphereai.com
Project-URL: Documentation, https://docs.simosphereai.com/sct/integrations/langchain
Project-URL: Repository, https://gitlab.simo-online.com/simosphereos/secure-compact-tokenization
Project-URL: Changelog, https://gitlab.simo-online.com/simosphereos/secure-compact-tokenization/-/blob/main/CHANGELOG.md
Author-email: SIMO GmbH <info@simo-online.com>
License-Expression: MIT
License-File: LICENSE
Keywords: compliance,dsgvo,gdpr,langchain,langgraph,middleware,pii,pseudonymization,tokenization
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Security :: Cryptography
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: langchain-core>=1.0
Requires-Dist: langchain>=1.3.2
Requires-Dist: sct-client>=2.2.0
Provides-Extra: dev
Requires-Dist: langchain-openai>=1.0; extra == 'dev'
Requires-Dist: langchain-tests>=0.3; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Requires-Dist: tiktoken>=0.7; extra == 'dev'
Provides-Extra: openai
Requires-Dist: langchain-openai>=1.0; extra == 'openai'
Requires-Dist: tiktoken>=0.7; extra == 'openai'
Description-Content-Type: text/markdown

# langchain-sct

**Reversible** PII pseudonymization for [LangChain 1.0](https://docs.langchain.com) —
a keyed, DSGVO/BaFin-grade **superset** of the built-in `PIIMiddleware`.

LangChain's own privacy tools are destructive or fake-reversible: `PIIMiddleware`'s
`redact`/`mask`/`hash` strategies are one-way, and `PresidioReversibleAnonymizer`
keeps a plaintext `mapping.json`. So the agent's final answer contains
`[REDACTED_email]` instead of the real name/IBAN.

`langchain-sct` closes the loop. It pseudonymizes PII into reversible
[SCT](https://sct.simosphereai.com) tokens **before** the model call and
**re-identifies** them **after** — the LLM never sees plaintext PII, yet the
user gets a fully usable, re-identified answer. Keying is AES-256-GCM / FF3-1
FPE with tenant key custody and audit, server-side.

## Install

```bash
pip install langchain-sct
```

## Flagship: `SCTPseudonymizationMiddleware`

One line into `create_agent` gives any agent reversible PII protection with **zero**
user code change:

```python
from langchain.agents import create_agent
from langchain_sct import SCTPseudonymizationMiddleware

agent = create_agent(
    "openai:gpt-5.5",
    tools=[...],
    middleware=[SCTPseudonymizationMiddleware(api_key="sct_...")],
)

# The LLM sees "<PERSON_0> at <EMAIL_1>"; the caller gets the real values back.
agent.invoke({"messages": [("user", "Email Max Mustermann at max@example.com")]})
```

| | built-in `PIIMiddleware` | `SCTPseudonymizationMiddleware` |
|---|---|---|
| Strategy | redact / mask / hash | **pseudonymize (reversible)** |
| Final answer | `[REDACTED_email]` | real value re-identified |
| Reverse the loop | impossible (one-way) | keyed `de-pseudonymize` |
| Entity coverage | 5 regex types (email, credit_card, ip, mac_address, url) | 13 DE/EU NER + regex types |
| Key custody | — | AES-256-GCM / FF3-1, audited |

### Configuration

```python
SCTPseudonymizationMiddleware(
    api_key="sct_...",                     # or set SCT_API_KEY
    strategy="pseudonymize",               # default reversible
    strategies={"CREDIT_CARD": "block"},   # per-type override (fail-fast)
    encryption_method="fpe-ff1",           # format-preserving tokens
    apply_to_input=True,
    apply_to_tool_results=False,
)
```

`strategy="block"` (or a per-type `block`) raises `SCTPIIDetectionError` in
`before_model` — the run fails before any content reaches the LLM.

## Portable LCEL: `with_sct_pseudonymization`

Middleware only fires inside `create_agent`. For plain LCEL chains (any provider,
any LangChain 0.x/1.x), wrap the chain instead:

```python
from langchain_sct import with_sct_pseudonymization

chain = with_sct_pseudonymization(prompt | llm | StrOutputParser(), client=sct)
answer = chain.invoke("Please email Max Mustermann at max@example.com")
```

The reversible token map is carried through the chain as the `encryption_key` via
a `RunnableParallel` fork — no plaintext mapping table ever leaves the SCT
boundary. `sct_pseudonymize` / `sct_reidentify` are exported for hand-composed
chains.

## Token compression: `SCTCompressionMiddleware`

The second differentiator — **deterministic** tool-output compression. LangChain's
own options for bulky observations are blind truncation (dumb) or a
summarizer-LLM call (an extra, non-deterministic model round-trip, explicitly
risky for finance/legal/health). SCT compresses via `POST /tokenizer/compress`:
format-aware, deterministic, `never_worse` (tiktoken-verified never to cost more
tokens than the raw input), **zero** extra LLM call.

```python
from langchain_sct import SCTCompressionMiddleware

agent = create_agent(
    "openai:gpt-5.5",
    tools=[search, run_tests],
    middleware=[SCTCompressionMiddleware(api_key="sct_...", min_chars=200)],
)
```

It overrides `wrap_tool_call`: the tool runs, then its `ToolMessage` content is
compressed on the way back into context. Each compressed message carries its
per-request `savings_pct` / `tier` in `response_metadata["sct_compression"]` —
the billing envelope travels with the message, not on the shared instance.

## RAG: `SCTRetriever` + `SCTPseudonymizingDocumentTransformer`

Extend "PII never enters context" to retrieval — RAG's biggest leak surface.
Retrieved chunks are pseudonymized before they reach the prompt; pair with the
middleware/anonymizer to re-identify the answer with the stamped key.

```python
from langchain_sct import SCTRetriever

retriever = SCTRetriever.from_api_key(vectorstore.as_retriever(), api_key="sct_...")
# Each returned Document has token-only page_content + metadata["sct_encryption_key"].
```

`SCTPseudonymizingDocumentTransformer` is usable standalone at ingest time
(`transform_documents` / `atransform_documents`).

## Presidio migration: `SCTReversibleAnonymizer`

Same `anonymize()` / `deanonymize()` surface as `langchain_experimental`'s
`PresidioReversibleAnonymizer`, but reversibility is keyed crypto — no plaintext
`mapping.json` crown-jewel to leak, and FF3-1 tokens stay format-preserving.
Migrate by changing one import:

```python
from langchain_sct import SCTReversibleAnonymizer

anonymizer = SCTReversibleAnonymizer(api_key="sct_...")
clean = anonymizer.anonymize("Email Max Mustermann at max@example.com")
restored = anonymizer.deanonymize(llm_answer)
```

## Billed endpoint: `ChatSCT` + `SCTUsageCallback`

`ChatSCT` is a `BaseChatOpenAI` subclass pinned to the SCT api-gateway's
OpenAI-compatible `/v1` surface — tenant/API-key headers injected, metering on by
default. Requires the `openai` extra:

```bash
pip install "langchain-sct[openai]"
```

```python
from langchain_sct import ChatSCT

llm = ChatSCT(model="gpt-5.5", api_key="sct_...", tenant_id="acme")
```

`SCTUsageCallback.on_llm_end` feeds each call's `usage_metadata` into an SCT
metering sink (wire your Stripe push there). A custom gateway `base_url` disables
OpenAI's streamed usage, so the callback carries a `tiktoken` fallback for exact
local counts.

## Async

Every surface has a native async path: the middleware implements
`abefore_model` / `aafter_model` / `awrap_tool_call` on `AsyncSCTClient`, the
runnables accept an `async_client=` for `ainvoke` / `abatch`, and the anonymizer
and document transformer expose `aanonymize` / `adeanonymize` /
`atransform_documents`.

## How it works

- `before_model` → `POST /pseudonymize` (`auto_detect_pii=true`); the returned
  `encryption_key` is carried in **agent state** (a private field), not on the
  middleware instance, so concurrent runs never share keys.
- `after_model` → `POST /de-pseudonymize` with that key, restoring the real
  values in the returned `AIMessage`.
- One key is reused across a conversation (bring-your-own-key on later turns) so
  re-identification is a single call.

## Benchmarks

The point isn't throughput — it's whether the agent's **final answer is usable**.
All three tools keep plaintext PII out of the model context; only SCT gives the
caller a re-identified answer back, and only SCT does it without a plaintext
mapping table.

Input in every row: `"Email Max Mustermann at max@example.com about IBAN
DE89370400440532013000"`.

| | What the **LLM** sees | What the **caller** gets back | Reversible? | Secret at rest |
|---|---|---|---|---|
| built-in `PIIMiddleware` (`redact`) | `Email [REDACTED_email]...` | `Email [REDACTED_email] about [REDACTED_...]` — **destroyed** | no (one-way) | none needed |
| built-in `PIIMiddleware` (`hash`/`mask`) | `Email a1b2c3…` / `****@****` | still hashed/masked — **not usable** | no | none |
| `PresidioReversibleAnonymizer` | `Email <PERSON> at <EMAIL_ADDRESS>` | real values **if** you call `deanonymize()` | yes | **plaintext `mapping.json`** on disk — the crown jewel |
| **`SCTPseudonymizationMiddleware`** | `Email <PERSON_0> at <EMAIL_1> about <IBAN_2>` | `Email Max Mustermann at max@example.com about DE89370400440532013000` — **fully re-identified, automatically** | yes (keyed) | ciphertext only; key custody + audit server-side, no plaintext map |

Concretely, side by side:

```python
# built-in PIIMiddleware — the answer is dead on arrival
create_agent(model, middleware=[PIIMiddleware("email", strategy="redact")])
# -> AIMessage("I've emailed [REDACTED_email].")   # useless to the user

# PresidioReversibleAnonymizer — reversible, but the mapping is plaintext on disk
anon = PresidioReversibleAnonymizer()
anon.anonymize("max@example.com")     # writes {"<EMAIL_ADDRESS>": "max@example.com"} to mapping.json
# a leaked mapping.json re-identifies every past run at once

# SCTPseudonymizationMiddleware — reversible AND keyed, no plaintext map, one line
create_agent(model, middleware=[SCTPseudonymizationMiddleware(api_key="sct_...")])
# -> AIMessage("I've emailed Max Mustermann at max@example.com.")  # usable, auto re-identified
```

### Token-compression benchmark

`SCTCompressionMiddleware` is measured against LangChain's two options for bulky
tool observations:

| Approach | Determinism | Extra LLM call | Guarantee |
|---|---|---|---|
| Blind truncation | deterministic | none | loses information silently |
| `SummarizationMiddleware` (summarizer LLM) | **non-deterministic** | **yes** (extra round-trip) | none — can hallucinate, risky for finance/legal/health |
| **`SCTCompressionMiddleware`** | **deterministic** | **none** | `never_worse` — tiktoken-verified never to cost more tokens than the raw input |

Each compressed `ToolMessage` reports its realized `savings_pct` / `tier` in
`response_metadata["sct_compression"]`, so the win is measured per call, not
assumed.

## DSGVO / GDPR nuance (read before you ship to a regulated tenant)

Reversibility is the feature — and the caveat. Be precise about what SCT does and
does not change legally:

- **Pseudonymized ≠ anonymized.** Under Art. 4(5) DSGVO / GDPR, reversibly
  pseudonymized data is *still personal data* (Recital 26) because the key can
  restore it. SCT reduces risk and enforces data minimization toward the LLM
  provider; it does **not** take the data out of scope. Do not market it as
  "anonymization." If you need true anonymization (out of scope), use a
  destructive strategy (`PIIMiddleware` redact/hash) instead — accepting that
  the answer is no longer re-identifiable.
- **Key custody is the control that matters.** The whole security argument over
  Presidio's `mapping.json` is that SCT never persists a plaintext token→value
  map: re-identification requires the tenant key, held server-side with audit.
  Treat the SCT API key (`sct_...`) and the per-conversation `encryption_key` as
  the crown jewels — scope them per tenant, rotate them, and keep them out of
  logs and out of the model context (the middleware already keeps the key in
  private agent state, never on the instance and never in a message).
- **Audit + purpose limitation.** Every de-pseudonymize is a re-identification
  event; log who triggered it and why. Set retention on encryption keys — once a
  conversation's key is destroyed, its tokens are cryptographically
  irreversible, which is a clean deletion story for erasure requests (Art. 17).
- **Deny/block on low-confidence entities for regulated tenants.** Auto-detect
  can miss or mis-type an entity. For BaFin/finance/health tenants, prefer a
  `block` strategy on the high-stakes types so an undetected or low-confidence
  hit **fails the run** rather than leaking to the model:

  ```python
  SCTPseudonymizationMiddleware(
      api_key="sct_...",
      strategies={"IBAN": "block", "CREDIT_CARD": "block", "HEALTH": "block"},
  )
  ```

  `block` raises `SCTPIIDetectionError` in `before_model` — nothing reaches the
  LLM. Fail closed, not open.
- **Not legal advice.** This documents SCT's technical behavior; your DPO owns
  the DSGVO assessment (DPIA, records of processing, DPA with the model provider).

## License

MIT © SIMO GmbH — see [LICENSE](LICENSE).
