Metadata-Version: 2.4
Name: iwa-monitor
Version: 0.3.1
Summary: Behavioral integrity layer for AI systems during training and inference
Author-email: signAI <umarjanjua@live.com>
License: MIT
Project-URL: Homepage, https://github.com/umarjanjua/signai
Project-URL: Repository, https://github.com/umarjanjua/signai
Project-URL: Documentation, https://github.com/umarjanjua/signai#readme
Keywords: ai-security,ml-security,adversarial-ml,model-monitoring,integrity
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.24
Requires-Dist: scikit-learn>=1.3
Requires-Dist: requests>=2.31.0
Provides-Extra: torch
Requires-Dist: torch>=2.1; extra == "torch"
Requires-Dist: torchvision>=0.16; extra == "torch"
Provides-Extra: server
Requires-Dist: fastapi>=0.115; extra == "server"
Requires-Dist: uvicorn[standard]>=0.30; extra == "server"
Requires-Dist: pydantic>=2.8; extra == "server"
Requires-Dist: python-multipart>=0.0.9; extra == "server"
Provides-Extra: postgres
Requires-Dist: psycopg2-binary>=2.9.0; extra == "postgres"

# signAI

Runtime behavioral integrity monitoring for PyTorch models.

signAI detects adversarial attacks, poisoned updates, and anomalous inputs at training and inference time — without sending model weights, raw inputs, or gradients anywhere. Only compact behavioral vectors are scored.

## What is in this repo

- `signai/client/` — SDK that ships on PyPI as [`iwa-monitor`](https://pypi.org/project/iwa-monitor/). Handles feature extraction, calibration, saving, loading, and scoring.
- `signai_server/` — daemon server that ships as a Docker image. Receives behavioral vectors from the SDK, fits detectors, and stores artifacts. Never shipped on PyPI.
- `signai/core/` — calibration engine and scoring algorithms. Private IP bundled inside the daemon Docker image.

## Install

```bash
pip install iwa-monitor
```

With PyTorch support:

```bash
pip install "iwa-monitor[torch]"
```

## How It Works

The SDK runs entirely on the customer machine. It uses PyTorch hooks to extract behavioral signals from the model, then sends compact float vectors to the signAI daemon for calibration and scoring. The daemon never sees model weights, raw inputs, or gradients.

```
Your machine                          signAI daemon (localhost:7731)
─────────────────────────────         ──────────────────────────────
model forward/backward pass
      ↓
feature extraction (hooks)
      ↓
s, z float vectors           →        fit detector / score
                             ←        score, flagged, tau
```

## Quick Start

### Step 1 — Start the daemon

```bash
docker run -p 7731:7731 signai/daemon
```

The SDK auto-discovers the daemon on `localhost:7731`. No configuration needed.

### Step 2 — Calibrate

```python
import signai

m = signai.attach(model, num_classes=10, monitor_id="my-model", device="cuda")
m.calibrate(clean_loader, device="cuda", phase="inference", calib_batches=200)
m.save("./integrity.json")
```

### Step 3 — Score at inference time

```python
import signai

m = signai.load(model, artifact="./integrity.json", device="cuda")

for x, y in test_loader:
    result = m.score_inference(x, y)
    if result.flagged:
        route_to_fallback(x)
```

### Step 4 — Score during training

```python
m = signai.load(model, artifact="./integrity.json", device="cuda")

for x, y in train_loader:
    optimizer.zero_grad()
    logits = model(x)
    loss = criterion(logits, y)
    loss.backward()
    optimizer.step()

    result = m.score_training(logits, loss)
    if result.flagged:
        quarantine_update(result)
```

## Deployment Modes

### Local artifact (no server needed for scoring)

Calibrate once with the daemon, save the artifact, then score locally without the daemon running:

```python
m = signai.load(model, artifact="./integrity.json", device="cuda")
result = m.score_inference(x, y)
```

### Self-hosted daemon

```python
m = signai.attach(
    model,
    num_classes=10,
    endpoint="http://signai.internal:7731",
    api_key="your-key",
    monitor_id="prod-model",
    device="cuda",
)
```

### Air-gapped

Same as self-hosted — point `endpoint` at your internal daemon address.

## Supported Models

Works with any PyTorch `nn.Module`:

- Standard CNNs, MLPs, RNNs
- HuggingFace Transformers (`input_ids`/`attention_mask` dict inputs, `.logits` outputs)
- Vision Transformers (ViT)
- Custom architectures — hooks are auto-selected from leaf modules

## Calibration Phases

| Phase | Detects | When to calibrate |
|---|---|---|
| `inference` | adversarial inputs, distribution shift | on clean test/validation data |
| `training` | poisoned updates, gradient manipulation | during a clean warmup training run |

## SDK Reference

```python
import signai

# Attach without loading an artifact (for calibration)
m = signai.attach(model, num_classes=10, monitor_id="...", device="cuda")

# Load a saved artifact (for scoring)
m = signai.load(model, artifact="./integrity.json", device="cuda")

# Calibrate (requires daemon)
m.calibrate(loader, device="cuda", phase="inference", calib_batches=200)
m.calibrate(loader, device="cuda", phase="training", optimizer=opt, criterion=loss_fn, warmup_steps=200)

# Save artifact locally
m.save("./integrity.json")

# Score
result = m.score_inference(x, y)   # returns MonitorResult
result = m.score_training(logits, loss)

# MonitorResult fields
result.score     # float — Mahalanobis distance
result.flagged   # bool — True if score > tau
result.tau       # float — calibration threshold
result.phase     # "inference" or "training"
result.error     # str or None
```

## CLI

```bash
# Check daemon status and usage
signai status

# Apply a license key
signai license <key>

# Show version
signai version
```

## Daemon API

The daemon exposes a REST API on port 7731:

- `GET /health`
- `POST /v1/calibrate/start`
- `POST /v1/calibrate/push`
- `POST /v1/calibrate/commit`
- `POST /v1/score/inference`
- `POST /v1/score/training`
- `POST /v1/score/batch`
- `GET /v1/artifacts/{monitor_id}`
- `GET /v1/artifacts/{monitor_id}/download`
- `DELETE /v1/artifacts/{monitor_id}`
- `GET /v1/usage`
- `POST /v1/license`

Pro/Enterprise endpoints:

- `GET /v1/history/{monitor_id}`
- `POST /v1/notify/configure`
- `GET /v1/audit/export`
- `GET /v1/status`
- `GET /v1/monitors` — enterprise only

## Editions

| Edition | Included |
|---|---|
| `community` | SDK, daemon, local artifact scoring, 10 calibration runs/month |
| `pro` | + history, notifications, audit export, status dashboard |
| `enterprise` | + Postgres, multi-node, unlimited runs |

License keys are applied via the daemon:

```bash
signai license <your-key>
```

## Privacy

Feature extraction runs on the customer machine. The daemon receives only `s` and `z` float vectors — no weights, no raw inputs, no gradients. Raw vectors are discarded after scoring. History stores only `{ts, score, flagged, phase}`.

## License

MIT
