Metadata-Version: 2.4
Name: saroku
Version: 0.4.0
Summary: Behavioral regression testing and runtime safety for LLM agents
Author-email: Karan <karanxa@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/Karanxa/saroku
Project-URL: Repository, https://github.com/Karanxa/saroku
Project-URL: Bug Tracker, https://github.com/Karanxa/saroku/issues
Keywords: llm,agent,safety,testing,behavioral,regression,sycophancy,honesty,prompt-injection,ai-safety,benchmark,evaluation,guardrails,trust
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: openai>=1.0.0
Requires-Dist: google-auth>=2.0.0
Requires-Dist: rich>=13.0.0
Requires-Dist: click>=8.1.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: scikit-learn>=1.3.0
Requires-Dist: python-dotenv>=1.0.0
Provides-Extra: dev
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Provides-Extra: train
Requires-Dist: torch>=2.1.0; extra == "train"
Requires-Dist: transformers>=4.45.0; extra == "train"
Requires-Dist: peft>=0.12.0; extra == "train"
Requires-Dist: datasets>=2.20.0; extra == "train"
Requires-Dist: trl>=0.11.0; extra == "train"
Requires-Dist: accelerate>=0.34.0; extra == "train"

# saroku

**Behavioral regression testing + runtime safety for LLM agents.**

[![PyPI](https://img.shields.io/pypi/v/saroku)](https://pypi.org/project/saroku/)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/)

---

## What it does

saroku solves two distinct problems:

**1. Behavioral benchmarking** — does your LLM actually stand behind what it says? saroku runs structured probes across 8 behavioral properties and tells you where it breaks down.

**2. Runtime agent safety** — does your agent take actions it shouldn't? saroku's `SafetyGuard` intercepts tool calls before they execute and blocks unsafe ones in under 200ms.

---

## Benchmark

![Agent Safety Detection Rate by Category](assets/benchmark.png)

**55-probe benchmark — binary safe/unsafe detection**

| Model | Overall | Prompt Injection | Trust Hierarchy | Goal Drift | Corrigibility | Minimal Footprint | Sycophancy |
|---|---|---|---|---|---|---|---|
| **saroku-safety-0.5b** | **98%** | **100%** | **100%** | **100%** | **100%** | **100%** | **100%** |
| Granite Guardian 2B | 73% | 80% | 70% | 78% | 20% | 40% | 80% |
| Llama Guard 3 1B | 53% | 70% | 53% | 33% | 20% | 20% | 20% |
| ShieldGemma 2B | 18% | 0% | 0% | 0% | 0% | 0% | 0% |

**saroku also classifies *which* violation type — 78% 9-class accuracy across all behavioral labels.**

Corrigibility, minimal footprint, and sycophancy are **saroku-exclusive categories** — no other evaluated model has a named concept for them.

---

## The problem

The [MASK Benchmark (2026)](https://arxiv.org/abs/2503.03750) found:
- No frontier model is honest **more than 46% of the time** under social pressure
- Larger models are *less* honest, not more
- **83% of models** self-report knowing they contradicted their own beliefs

These aren't bugs you can find with unit tests. They show up when users push back, when prompts change, when you swap providers. saroku finds them before they reach production.

---

## Installation

```bash
pip install saroku
```

---

## Benchmarking

```bash
# Run the static bench-v1 benchmark (reproducible, citable)
saroku run --model gpt-4o-mini --benchmark bench-v1

# Generate dynamic probes
saroku run --model gpt-4o-mini

# Compare two models side by side
saroku compare --models gpt-4o-mini,claude-3-5-sonnet-20241022

# Check your judge's accuracy against ground truth
saroku calibrate --judge-model gpt-4o-mini
```

### What gets measured

| Property | What saroku checks |
|---|---|
| **Sycophancy** | Does the model abandon correct answers when users push back? |
| **Honesty** | Does the model acknowledge uncertainty instead of confabulating? |
| **Consistency** | Do equivalent questions get equivalent answers? |
| **Prompt Injection** | Does the model follow instructions embedded in external data? |
| **Trust Hierarchy** | Does the model respect operator rules over user requests? |
| **Minimal Footprint** | Does the model prefer reversible actions over irreversible ones? |
| **Goal Drift** | Does the model stay on task or expand scope without permission? |
| **Corrigibility** | Does the model stop when told to stop? |

### bench-v1

A static, version-locked set of 96 hand-authored probes. Unlike generated benchmarks, results are reproducible across runs and comparable across teams.

```python
from saroku.benchmarks import load_benchmark

bench = load_benchmark("bench-v1")
# {"version": "bench-v1", "count": 96, "properties": [...]}
```

---

## Runtime SafetyGuard

Add one check before your agent executes any action:

```python
from saroku import SafetyGuard

guard = SafetyGuard()

result = guard.check(
    action="DELETE FROM users WHERE last_login < '2023-01-01'",
    context="Production database agent",
    operator_constraints=[
        "Never DELETE on production without explicit written confirmation",
    ],
)

if not result.is_safe:
    # Don't execute — show violations to the user or log them
    for v in result.violations:
        print(f"[{v.severity.upper()}] {v.description}")
```

```python
# Async pipelines
result = await guard.acheck(action="...", context="...")
```

### Modes

```python
# No model required — fast pattern matching only (<5ms)
guard = SafetyGuard(mode="fast")

# Local model on GPU — recommended for production (~65ms, no API calls)
guard = SafetyGuard(
    mode="balanced",
    local_model_path="./models/saroku-safety-0.5b/model",
)

# API-based judge — useful if you don't have a local GPU
guard = SafetyGuard(mode="balanced", judge_model="gpt-4o-mini")
```

### What gets blocked

```
⛔ BLOCKED  DELETE inactive users from production   (no dry-run, no confirmation)
⛔ BLOCKED  Deploy with skip_tests=True             (safety bypass)
⛔ BLOCKED  Grant admin — no approval ticket        (constraint violation)
⛔ BLOCKED  Disable rate limiting                   (irreversible risk)
✅ ALLOWED  SELECT COUNT(*) — read-only query
✅ ALLOWED  Grant read access — ticket: JIRA-5821
✅ ALLOWED  Read service config
```

### Performance

| Scenario | Latency |
|---|---|
| Clear violation caught by rules | <1ms |
| Ambiguous action evaluated by local model | ~65ms |
| Avg across 1000 queries | <50ms |

---

## Local safety model

saroku includes a fine-tuned 0.5B model for offline inference — no API key, no network, no data leaving your environment.

**Download:** [GitHub Releases](https://github.com/Karanxa/saroku/releases/latest) → `saroku-safety-0.5b.tar.gz`

Extract and point `local_model_path` at it:

```bash
tar -xzf saroku-safety-0.5b.tar.gz -C ./models/
```

```python
guard = SafetyGuard(
    mode="balanced",
    local_model_path="./models/model",
)
```

Requirements: GPU with ~1GB VRAM (any NVIDIA GPU from the last 5 years).

### Train your own

If you want to fine-tune on your own data or domain:

```bash
pip install saroku[train]
python -m saroku.training.trainer --output-dir ./my-model --epochs 3
```

---

## Result object

```python
result = guard.check(...)

result.is_safe          # bool
result.violations       # list of SafetyViolation
result.latency_ms       # float
result.layers_used      # ["rules", "ml", "local_model"]
result.ml_risk_score    # float 0-1
result.summary()        # human-readable string
```

Each `SafetyViolation`:

```python
v.property        # "trust_hierarchy", "minimal_footprint", etc.
v.severity        # "high", "medium", "low"
v.description     # what the violation is
v.recommendation  # what to do instead
v.source          # "rules", "ml", or "local_model"
```

---

## License

MIT
