Metadata-Version: 2.4
Name: zero_harm_ai_detectors
Version: 0.2.10
Summary: Privacy & content safety detection with heuristic and AI modes
Author-email: Zero Harm AI <contact@zeroharm.ai>
License: MIT
Project-URL: Homepage, https://scalapps.com
Project-URL: Documentation, https://scalapps.com/docs/ai_safety/PUBLIC_API_GUIDE.md
Project-URL: License, https://scalapps.com/docs/LICENSE.txt
Project-URL: Commercial, https://scalapps.com/docs/LICENSE-COMMERCIAL.txt
Keywords: pii,privacy,detection,secrets,security,ai,nlp
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: LICENSE-COMMERCIAL.md
Requires-Dist: regex>=2022.1.18
Provides-Extra: ai
Requires-Dist: transformers>=4.30.0; extra == "ai"
Requires-Dist: torch>=2.0.0; extra == "ai"
Requires-Dist: spacy<3.8.0,>=3.7.0; python_version < "3.10" and extra == "ai"
Requires-Dist: spacy>=3.8.0; python_version >= "3.10" and extra == "ai"
Requires-Dist: sentencepiece; extra == "ai"
Requires-Dist: protobuf; extra == "ai"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: flake8>=6.0.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Dynamic: license-file

# Zero Harm AI Detectors

**Privacy & content safety detection with heuristic default and optional AI config.**

[![PyPI version](https://badge.fury.io/py/zero-harm-ai-detectors.svg)](https://pypi.org/project/zero-harm-ai-detectors/)
[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## One API, Two Execution Paths

| Feature | `detect(text)` | `detect(text, ai_config=...)` |
|---------|--------------|-----------|
| **Speed** | ⚡ 1–5ms | 🐢 50–200ms |
| **Email, Phone, SSN** | ✅ 95–99% | ✅ 95–99% |
| **Credit Card (Luhn)** | ✅ 99% | ✅ 99% |
| **Secrets / API Keys** | ✅ 95% | ✅ 95% |
| **Person Names** | ⚠️ 20–60% (tiered) | ✅ 85–95% |
| **Locations** | ❌ | ✅ 80–90% |
| **Organisations** | ❌ | ✅ 75–85% |
| **Harmful Content** | ✅ Pattern-based | ✅ Contextual AI |
| **Extra dependencies** | None | `transformers`/`torch` and/or `spacy` |

## Installation

```bash
# Heuristic mode only (fast, no ML dependencies)
pip install zero_harm_ai_detectors

# With AI mode (~2 GB model download on first use)
pip install 'zero_harm_ai_detectors[ai]'
```

On Python 3.9, the `ai` extra installs the spaCy 3.7 line for compatibility.

## Quick Start

```python
from zero_harm_ai_detectors import detect, AIConfig

# Heuristic mode (default) — fast, great for structured data
result = detect("Email: john@example.com, SSN: 123-45-6789")
print(result.redacted_text)
# → Email: [REDACTED_EMAIL], SSN: [REDACTED_SSN]

# AI mode — enabled when ai_config is provided
result = detect("Contact John Smith at Microsoft in NYC", ai_config=AIConfig())
print(result.detections)
# → [Detection(PERSON, ...), Detection(ORGANIZATION, ...), Detection(LOCATION, ...)]
```

## Detection Result

Both modes return an identical `DetectionResult`:

```python
result = detect(text)  # heuristic
result_ai = detect(text, ai_config=AIConfig())  # ai

result.original_text   # str   — original input
result.redacted_text   # str   — sensitive content replaced
result.detections      # list  — List[Detection]
result.mode            # str   — "heuristic" or "ai"
result.harmful         # bool  — harmful content found
result.severity        # str   — "none" | "low" | "medium" | "high"
result.harmful_scores  # dict  — per-category scores

result.to_dict()       # full dict with all fields
result.get_pii()       # List[Detection] — PII only
result.get_secrets()   # List[Detection] — secrets only
```

## Target Selection

```python
from zero_harm_ai_detectors import detect, DetectTarget

# Default = all targets
detect(text)

# PII only
detect(text, targets=DetectTarget.PII)

# Secrets only
detect(text, targets=DetectTarget.SECRET)

# Harmful only
detect(text, targets=DetectTarget.HARMFUL)

# Combine
detect(text, targets=DetectTarget.PII | DetectTarget.SECRET)
```

## PII Detection

### Structured data (heuristic, 95–99% accuracy)

```python
from zero_harm_ai_detectors import (
    detect_emails,
    detect_phones,
    detect_ssns,
    detect_credit_cards,
    detect_bank_accounts,
    detect_dob,
    detect_addresses,
)

emails = detect_emails("Contact alice@test.com or bob@example.org")
# → [Detection(EMAIL, 'alice@test.com', ...), Detection(EMAIL, 'bob@example.org', ...)]

cards = detect_credit_cards("Card: 4532-0151-1283-0366")
# → [Detection(CREDIT_CARD, '4532-0151-1283-0366', confidence=0.99)]  # Luhn validated
```

### Person names — heuristic mode (tiered confidence)

Heuristic-mode person name detection uses three tiers to balance recall against
false positives. Every `PERSON` detection carries `metadata["tier"]` and a
confidence value from the documented set `{0.20, 0.45, 0.60}`.

| Tier | Confidence | Trigger | Precision |
|------|-----------|---------|-----------|
| **1 — titled** | `0.60` | Name preceded by a title: `Mr`, `Mrs`, `Ms`, `Miss`, `Dr`, `Prof`, `Rev`, `Capt`, `Lt`, `Sgt`, `Cpl`, `Pvt` | High |
| **2 — context** | `0.45` | Name follows a context keyword: `Name:`, `Contact:`, `From:`, `To:`, `Cc:`, `Bcc:`, `Signed by:`, `Authored by:` | Medium |
| **3 — bare** | `0.20` | Plain `"First Last"` capitalised pair with no surrounding signal — high false-positive rate | Low |

Tiers 1 and 2 are checked first; any span they cover is suppressed from Tier 3,
so a titled name never appears as a duplicate bare match.

```python
from zero_harm_ai_detectors import detect_person_names_heuristic

detections = detect_person_names_heuristic(
    "Dr. Alice Brown arrived. Contact: Bob White. Meet James Green tomorrow."
)

for d in detections:
    print(f"{d.text!r:20}  tier={d.metadata['tier']:8}  confidence={d.confidence}")

# 'Dr. Alice Brown'     tier=titled    confidence=0.6
# 'Bob White'           tier=context   confidence=0.45
# 'James Green'         tier=bare      confidence=0.2
```

**Filtering by confidence threshold** is the recommended way to tune precision
vs. recall for your use case:

```python
# High-precision only: titled and context names
precise = [d for d in detections if d.confidence >= 0.45]

# Maximum recall: include bare matches too
all_names = detections  # confidence >= 0.20

# Inspect tier directly
titled_only = [d for d in detections if d.metadata.get("tier") == "titled"]
```

> **Tip:** For reliable person-name detection (85–95% accuracy) use
> `detect(text, ai_config=AIConfig())` instead. The heuristic tiers are best suited to
> structured forms and templated text where titles or context keywords are
> consistently present.

### Names, locations, organisations (AI mode only)

```python
from zero_harm_ai_detectors import AIConfig

result = detect("Dr. Jane Wilson visited Stanford University in Palo Alto", ai_config=AIConfig())

for d in result.detections:
    print(f"{d.type}: {d.text} ({d.confidence:.0%})")
# PERSON: Dr. Jane Wilson (92%)
# ORGANIZATION: Stanford University (88%)
# LOCATION: Palo Alto (85%)
```

## Secrets Detection

Three-tier detection, always uses heuristic matching (95%+ accuracy):

```python
from zero_harm_ai_detectors import detect_secrets_heuristic

secrets = detect_secrets_heuristic("""
    OPENAI_KEY = "sk-1234567890abcdef1234567890abcdef"
    AWS_KEY    = "AKIAIOSFODNN7EXAMPLE"
    GH_TOKEN   = "ghp_ABCDEFGHIJKLMNOPQRSTUVWXYZabcd1234"
""")
```

**Supported patterns:**
OpenAI · AWS (access + secret) · GitHub · Stripe · Google · Slack ·
Twilio · SendGrid · npm · PyPI · Anthropic · Generic secrets (context + entropy)

## Harmful Content Detection

```python
from zero_harm_ai_detectors import DetectTarget

result = detect(text, targets=DetectTarget.HARMFUL)

print(result.harmful)         # True / False
print(result.severity)        # "none" | "low" | "medium" | "high"
print(result.harmful_scores)  # {"insult": 0.6, "threat_phrases": 0.8, ...}
```

**Severity rules:**

| Condition | Severity |
|-----------|----------|
| `identity_hate` found | **high** |
| Explicit threat phrase | **high** |
| 2+ threat words or 6+ total matches | **high** |
| 1 threat word or 4+ total matches | **medium** |
| 2+ obscene terms | **medium** |
| Any other match | **low** |

## Redaction Strategies

```python
text = "Email: john@example.com"

detect(text, redaction_strategy="token")     # Email: [REDACTED_EMAIL]
detect(text, redaction_strategy="mask_all")  # Email: ****************
detect(text, redaction_strategy="mask_last4")# Email: ************.com
detect(text, redaction_strategy="hash")      # Email: [HASH:a1b2c3d4e5f6]
```

## Selective Detection

```python
from zero_harm_ai_detectors import DetectTarget

# PII only
result = detect(text, targets=DetectTarget.PII)

# Secrets only
result = detect(text, targets=DetectTarget.SECRET)

# PII + Secrets
result = detect(text, targets=DetectTarget.PII | DetectTarget.SECRET)
```

## AI Configuration

```python
from zero_harm_ai_detectors import detect, AIConfig

config = AIConfig(
    ner_backend="transformers",  # higher NER accuracy
    ner_model="dslim/bert-base-NER",
    ner_threshold=0.8,   # higher = fewer false positives
    harmful_threshold=0.6,
    device="cuda",       # or "cpu"
)

result = detect(text, ai_config=config)
```

SpaCy NER option:

```python
config = AIConfig(
    ner_backend="spacy",
    spacy_model="en_core_web_sm",
)
result = detect(text, ai_config=config)
```

Backward compatibility:
- `AIConfig(backend="transformers")` still works as an alias for `ner_backend`.

Harmful detection behavior in AI path:
- Uses transformer harmful model when available.
- If unavailable/fails, falls back to heuristic harmful detection.
- Non-default `harmful_model` values emit a warning.

## Architecture

```
┌────────────────────────────────────────────────────────────┐
│      detect(text, targets=..., ai_config=...)              │
├──────────────────────────┬─────────────────────────────────┤
│ default (no ai_config)   │ ai_config provided              │
│ heuristic_detectors.py   │ ai_detectors.py                 │
│                          │ - structured PII via heuristic  │
│ - structured PII         │ - NER via transformers or spacy │
│ - secrets via heuristic  │ - harmful via transformer       │
│ - harmful via heuristic  │   with heuristic fallback       │
└──────────────────────────┴─────────────────────────────────┘
                         ↓
                    DetectionResult
```

## Integration Example — GitHub App PR Scanner

```python
from zero_harm_ai_detectors import detect, AI_AVAILABLE, AIConfig, DetectTarget

def scan_pr_diff(diff: str, is_paid_user: bool) -> dict:
    ai_config = AIConfig() if (is_paid_user and AI_AVAILABLE) else None

    result = detect(
        diff,
        targets=DetectTarget.PII | DetectTarget.SECRET,
        ai_config=ai_config,
    )

    blocking_types = {"API_KEY", "SECRET", "SSN", "CREDIT_CARD"}
    return {
        "has_issues":   len(result.detections) > 0,
        "should_block": any(d.type in blocking_types for d in result.detections),
        "detections":   result.to_dict()["detections"],
    }
```

## Performance

| Text length | Default (heuristic) | AI (`ai_config`) |
|-------------|-------|----|
| ~50 chars | 1–2ms | 50–100ms |
| ~500 chars | 2–3ms | 100–150ms |
| ~5 000 chars | 3–5ms | 150–200ms |
| Throughput | ~500/sec | ~5–10/sec |

## Testing

```bash
# All tests
pytest tests/ -v

# With coverage
pytest tests/ --cov=zero_harm_ai_detectors

# Skip AI tests (if dependencies not installed)
pytest tests/test_core_and_heuristic.py -v
```

## Licensing

The open-source core of this project is licensed under MIT.

- Open-source license: https://scalapps.com/docs/LICENSE
- Commercial licensing terms: https://scalapps.com/docs/LICENSE-COMMERCIAL

Commercial licensing is available for organizations that need contractual terms
such as support/SLA, warranty, indemnity, or other enterprise conditions.
Contact: `info@scalapps.com`

## Links

- **PyPI**: https://pypi.org/project/zero-harm-ai-detectors/
- **Issues**: https://scalapps.com
