Metadata-Version: 2.4
Name: llm-shield-proxy
Version: 1.0.2
Summary: Enterprise Zero-Egress Privacy Redaction Proxy Engine for LLMs
Author-email: Ninad Phalak <ninad.phalak@gmail.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/ninadphalak/LLM-Shield-Proxy
Project-URL: Repository, https://github.com/ninadphalak/LLM-Shield-Proxy
Project-URL: Bug Tracker, https://github.com/ninadphalak/LLM-Shield-Proxy/issues
Keywords: llm,pii,redaction,privacy,proxy,fastapi,streaming,soc2,hipaa
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi>=0.100.0
Requires-Dist: uvicorn>=0.22.0
Requires-Dist: httpx>=0.24.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pydantic-settings>=2.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: pytest-httpx>=0.24.0; extra == "dev"
Requires-Dist: pytest-mock>=3.10.0; extra == "dev"
Requires-Dist: openai>=1.0.0; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

# LLM-Shield-Proxy - Enterprise Privacy Redaction Engine

[![PyPI version](https://badge.fury.io/py/llm-shield-proxy.svg)](https://pypi.org/project/llm-shield-proxy/)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Python Version](https://img.shields.io/badge/python-3.9%2B-blue)](https://www.python.org/)

A zero-latency, zero-egress, streaming-safe PII redaction proxy for Enterprise LLMs.

**LLM-Shield-Proxy** is an open-source, zero-egress middleware proxy that intercepts OpenAI-compatible LLM API requests, redacts Personally Identifiable Information (PII) before it leaves your local infrastructure, and deterministically re-hydrates real-time SSE streaming responses without breaking stream latency.

Designed to unblock enterprise privacy compliance (**SOC 2 / HIPAA**).

Author & Core Maintainer: **Ninad Phalak** (`ninad.phalak@gmail.com`)

---

## 🏗️ Architecture & Data Flow

```mermaid
flowchart TD
    classDef client fill:#e0f2fe,stroke:#0284c7,stroke-width:2px,color:#0369a1,font-weight:bold;
    classDef proxyEngine fill:#f8fafc,stroke:#475569,stroke-width:2px,color:#0f172a,font-weight:bold;
    classDef piiSecurity fill:#fef2f2,stroke:#ef4444,stroke-width:2px,color:#991b1b,font-weight:bold;
    classDef vault fill:#fffbebe,stroke:#f59e0b,stroke-width:2px,color:#92400e,font-weight:bold;
    classDef upstream fill:#f3e8ff,stroke:#9333ea,stroke-width:2px,color:#6b21a8,font-weight:bold;

    UserApp["👤 User Application\n(OpenAI / LangChain SDK)"]:::client

    subgraph SecurityMoat ["🛡️ Zero-Egress Local Environment (Apache 2.0 Licensed)"]
        direction TD
        FastAPIProxy["⚡ FastAPI Catch-All Proxy\n(/{path:path})"]:::proxyEngine

        subgraph CascadeEngine ["🔒 Two-Tier PII Cascade Engine"]
            Tier1["Tier 1: Compiled Regex"]:::piiSecurity
            Tier2["Tier 2: Quantized ONNX NER"]:::piiSecurity
            Tier1 --> Tier2
        end

        VaultStore[("🔑 Session Vault Store\n(Deterministic Tokens)")]:::vault
        LookaheadBuffer["⏱️ Sliding-Window Lookahead Buffer\n(Prevent SSE Tag Leaks)"]:::proxyEngine
        Rehydrator["🔄 Stream Re-hydrator\n(Token -> Original Value)"]:::proxyEngine
    end

    UpstreamLLM["☁️ Upstream LLM Provider\n(OpenAI / Anthropic / vLLM)"]:::upstream

    %% Inbound Flow (Prompt Sanitization)
    UserApp -- "1. Inbound Raw Prompt Payload" --> FastAPIProxy
    FastAPIProxy -- "2. Scan Payload" --> Tier1
    Tier2 -- "3. Store Vault Keys" --> VaultStore
    Tier2 -- "4. Redacted JSON Payload" --> UpstreamLLM

    %% Outbound Flow (Streaming De-redaction)
    UpstreamLLM -. "5. Raw SSE Stream Deltas" .-> LookaheadBuffer
    LookaheadBuffer -- "6. Tag-Safe Assembly" --> Rehydrator
    Rehydrator <--> VaultStore
    Rehydrator -. "7. Sanitized Real-Time Stream" .-> UserApp

    style SecurityMoat fill:#f8fafc,stroke:#0284c7,stroke-width:2px,stroke-dasharray: 5 5,color:#0f172a
    style CascadeEngine fill:#ffffff,stroke:#cbd5e1,stroke-width:1px
```

### How It Works (The Data Flow)

#### 📥 Inbound (Prompt Sanitization)
1. **Intercept:** Your application sends a standard OpenAI / LangChain payload to `localhost:8000`.
2. **Cascade Redaction:** The proxy intercepts the JSON and routes text through a high-speed compiled Regex engine (SSNs, emails, credit cards), falling back to a local ONNX model for unstructured names.
3. **Vault Storage:** The original PII is mapped to a deterministic tag (e.g., `[PERSON_1]`) and stored locally in a TTL-backed session vault.
4. **Clean Egress:** A 100% sanitized payload is forwarded to OpenAI. OpenAI never sees your raw sensitive data.

#### 📤 Outbound (Streaming De-redaction)
1. **SSE Stream Intercept:** OpenAI streams the response back chunk-by-chunk via Server-Sent Events (SSE).
2. **Lookahead Buffer:** Because tags can be split across SSE chunks (e.g., `[PER` in chunk N and `SON_1]` in chunk N+1), the proxy's sliding-window buffer holds back unclosed brackets to prevent tag leaks.
3. **Re-hydration:** Once a tag is fully assembled, the proxy swaps the real data back from the local vault and streams the final, un-redacted text to the user's application in real-time.

---

## 🔒 Why LLM-Shield-Proxy? (Architectural Moats)

Why not just write a basic regex script? Basic regex scripts break on streaming responses, leak split tokens, and slow down your application. LLM-Shield-Proxy is purpose-built for production:

- **Streaming Safety:** Includes a proprietary **Sliding-Window Lookahead Buffer** that prevents Server-Sent Event (SSE) chunking leaks across split tokens (e.g., holding back `[PER` until `SON_1]` arrives).
- **Speed & Accuracy:** Leverages a **Two-Tier Cascade Engine** (sub-millisecond compiled Regex + Quantized ONNX NER) that catches unstructured names in ~10ms with zero cloud dependencies.
- **Context Preservation:** Uses a **Local TTL Session Vault** that maps PII to session-bound tokens (`[PERSON_1]`), enabling the LLM to retain full conversational context without receiving raw sensitive data.
- **Zero-Egress Security:** Runs 100% on your local infrastructure. Your raw data never leaves your server; only sanitized, redacted payloads reach upstream LLM providers.

---

## ⚡ Core Features

- **Zero Latency Streaming:** Sliding-window tag-safety buffer intercepts SSE streams delta-by-delta without buffering full requests or responses.
- **Zero Cloud / Zero Egress:** 100% local processing. No external API calls for PII detection.
- **Two-Tier PII Cascade Engine:**
  - **Tier 1 (Sub-millisecond Regex):** SSNs, Credit Cards, Email Addresses, Phone Numbers, IPv4/IPv6, API Keys.
  - **Tier 2 (NER Engine):** Person Names and unstructured entities.
- **Deterministic Re-Hydration Vault:** Swaps PII with session-bound tokens (e.g., `Sarah` -> `[PERSON_1]`). Maps back deterministically when the LLM streams responses. Supports request-scoped and session-scoped (`X-Session-ID`) vaults.
- **SOC 2 Structured Audit Logging:** Emits JSON structured audit logs for compliance monitoring.
- **Tier 2 (NER Engine):** A lightweight, quantized ONNX Named Entity Recognition (NER) model. It uses local AI to catch unstructured names and entities that slip past standard regex, executing in ~10ms without requiring heavy Python frameworks or cloud APIs.

---

## 🛠️ Quickstart

### Installation

Install the package from PyPI:

```bash
pip install llm-shield-proxy
```

### Configuration
Create a `.env` file in the root directory before starting the server.

```env
# Required upstream provider key (or pass via Authorization header)
OPENAI_API_KEY=sk-your-openai-key-here

# Optional configuration
PORT=8000
TELEMETRY_ENABLED=false
```

#### 1. Start the Proxy

Run the proxy locally via Docker or Uvicorn. No external database required for the open-source core.

```bash
uvicorn app.main:app --host 0.0.0.0 --port 8000
```
or via Docker Compose:
```bash
docker-compose up -d
```

#### 2. Update your Application (1-Line Change)

Point your existing OpenAI SDK `base_url` to your local LLM-Shield-Proxy instance.

```python
from openai import OpenAI

client = OpenAI(
    api_key="your-openai-api-key",
    base_url="http://localhost:8000/v1" # Point to LLM-Shield-Proxy
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "user", "content": "Contact Sarah Connor at sarah@example.com"}
    ],
    stream=True
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")
```

---

## ⚠️ Known Limitations

Transparency is critical for security tooling. Please be aware of the following current limitations:
- **Text Only:** The proxy does not currently scan or redact text embedded inside base64 image payloads (e.g., OpenAI Vision models).
- **Supported Languages:** The Tier-2 ONNX NER model is currently optimized for English-language entities. 
- **Non-Standard Streaming:** Designed for standard Server-Sent Events (SSE). Custom or proprietary streaming protocols may bypass the sliding-window buffer.

---

## 🧪 Testing

Run the full automated test suite:

```bash
py -m pytest tests/
```

---

## 🏢 Using LLM-Shield-Proxy in Production?

We are actively working with enterprise security teams to map out advanced compliance features. If your startup or organization is using LLM-Shield-Proxy to unblock LLM streaming or pass SOC 2/HIPAA audits, I would love to hear from you.

Email the core maintainer at ninad.phalak@gmail.com to share your feedback, request a feature, or feature your team as a case study.
