Metadata-Version: 2.4
Name: signai-sdk
Version: 0.4.12
Summary: Behavioral integrity layer for AI systems during training and inference
Author-email: signAI <umarjanjua@live.com>
License: AGPL-3.0-only
Project-URL: Homepage, https://github.com/umarjanjua/signai
Project-URL: Repository, https://github.com/umarjanjua/signai
Project-URL: Documentation, https://github.com/umarjanjua/signai#readme
Keywords: ai-security,ml-security,adversarial-ml,model-monitoring,integrity
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: OSI Approved :: GNU Affero General Public License v3
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.24
Requires-Dist: torch>=2.1
Requires-Dist: cryptography>=41.0
Requires-Dist: requests>=2.31
Requires-Dist: tqdm>=4.66
Provides-Extra: server
Requires-Dist: fastapi>=0.115; extra == "server"
Requires-Dist: uvicorn[standard]>=0.30; extra == "server"
Requires-Dist: pydantic>=2.8; extra == "server"
Requires-Dist: python-multipart>=0.0.9; extra == "server"
Requires-Dist: torchvision>=0.16; extra == "server"
Requires-Dist: scikit-learn>=1.3; extra == "server"
Requires-Dist: matplotlib>=3.7; extra == "server"
Provides-Extra: postgres
Requires-Dist: psycopg2-binary>=2.9.0; extra == "postgres"

# signAI

Runtime behavioral integrity monitoring for PyTorch models.

signAI adds a small monitoring layer around training and inference so you can detect suspicious model behavior in real time — without exposing model weights, raw inputs, or gradients. The production layer supports:

- Local artifact mode with no server
- Daemon mode (localhost:7731, auto-discovered by the SDK)
- Self-hosted server mode
- Air-gapped private deployment mode

## What signAI detects

signAI monitors a model's behavioral fingerprint rather than its accuracy. It catches anomalies such as:

- distribution shift in inputs or activations
- gradient manipulation and targeted poisoning during training
- unexpected output distribution changes at inference time

Detection is based on a conditional behavioral model (CBM): for each operating state S, signAI learns the expected behavioral response Z and flags deviations.

## What is in this repo

- `signai/core/`: research and scoring core
- `signai/core/extractors/`: S/Z feature extractors for classifiers, LLMs, and custom models
- `signai/core/detectors/`: detector registry (Mahalanobis, neural, association)
- `signai/client/`: production SDK
- `signai_server/`: standalone REST server for hosted and self-hosted scoring

## Install

SDK only, local mode:

```bash
pip install signai
```

SDK plus server dependencies:

```bash
pip install "signai[server]"
```

SDK plus server plus Postgres support:

```bash
pip install "signai[server,postgres]"
```

For local development in this repo:

```bash
python -m venv .venv
.venv\Scripts\activate
pip install -e ".[server]"
```

## Quick Start

### Local mode (any classifier)

```python
from signai import monitor

m = monitor.attach(model, num_classes=10)
m.calibrate(clean_loader, device="cuda", phase="inference", calib_batches=200)
m.save("./integrity.json")
```

Load and score:

```python
m = monitor.load(model, artifact="./integrity.json", device="cuda")

for x, y in test_loader:
    result = m.score_inference(x, y)
    if result.flagged:
        route_to_fallback(x)
```

### LLM / large model quick start

For models above 200M parameters or HuggingFace generative models, use `LLMExtractor` and `IntegrityMonitor` directly:

```python
from signai import IntegrityMonitor, LLMExtractor

extractor = LLMExtractor(llm)
m = IntegrityMonitor(llm, num_classes=None, extractor=extractor, detector_kind="v1")
m.calibrate_inference(clean_loader, calib_batches=200, device="cuda")
m.export_json("./integrity_llm.json")
```

Score LLM inference:

```python
m = IntegrityMonitor(llm, num_classes=None, extractor=LLMExtractor(llm), detector_kind="v1")
m.import_json("./integrity_llm.json")

for batch in eval_loader:
    result = m.score_input(batch["input_ids"], None, device="cuda")
```

### Remote mode

Start the server:

```bash
signai-server serve --host 0.0.0.0 --port 8000 --storage ./artifacts
```

Connect the SDK:

```python
from signai import monitor

m = monitor.attach(
    model,
    num_classes=10,
    endpoint="http://localhost:8000",
    api_key="",
    monitor_id="demo-model",
    device="cuda",
)
m.calibrate(clean_loader, device="cuda", phase="inference", calib_batches=200)
m.save("./integrity.json")
```

## Detector Kinds

signAI ships three detector algorithms. Choose by setting `detector_kind`:

| Kind | Algorithm | Best for |
|------|-----------|----------|
| `"v1"` | Conditional Mahalanobis (sklearn) | general purpose; fast; no GPU required |
| `"nn"` | Neural conditional monitor | complex activation geometry; higher accuracy |
| `"assoc"` | Blockwise association neural monitor | high-dimensional behavioral spaces |

Use `v1` as the default. Use `nn` or `assoc` when you need finer discrimination on complex models.

```python
from signai import IntegrityMonitor, ClassificationExtractor

# Neural detector
m = IntegrityMonitor(
    model,
    num_classes=10,
    extractor=ClassificationExtractor(model),
    detector_kind="nn",
)
```

## Extractor Kinds

signAI uses an extractor plugin system to support different model types. The right extractor is auto-selected when you use `monitor.attach()`:

| Extractor | Auto-selected when | Description |
|-----------|-------------------|-------------|
| `ClassificationExtractor` | ≤200M params, classification | CNN/ViT classifier; tracks gradient geometry and activation depth |
| `LLMExtractor` | >200M params or HF generative | Per-module L2 norm delta tracking; constant memory |

To override auto-selection:

```python
from signai import IntegrityMonitor, LLMExtractor

# Force LLMExtractor on a model below the size threshold
m = IntegrityMonitor(model, extractor=LLMExtractor(model), detector_kind="v1")
```

To implement a custom extractor for a novel architecture:

```python
from signai import SignatureExtractorBase
import numpy as np

class MyExtractor(SignatureExtractorBase):
    def prepare(self, example_batch): ...
    def reset_state(self): ...
    def extract_training(self, logits, loss) -> tuple[np.ndarray, np.ndarray]: ...
    def extract_inference(self, x, y, device="cpu", use_sensitivity=True) -> tuple[np.ndarray, np.ndarray]: ...
```

## Deployment Modes

### 1. Local artifact

```python
m = monitor.load(model, artifact="./integrity.json")
```

### 2. Daemon (localhost:7731)

The SDK auto-discovers a running `signai-server` on `localhost:7731`. No endpoint config needed:

```python
m = monitor.attach(model, num_classes=10)  # connects to daemon if running
```

### 3. Self-hosted

```python
m = monitor.load(
    model,
    endpoint="http://signai.internal:8000",
    api_key="...",
    monitor_id="my-model",
)
```

### 4. Air-gapped

```python
m = monitor.load(
    model,
    endpoint="http://10.0.1.5:8000",
    api_key="...",
    monitor_id="my-model",
)
```

## Training Loop Integration

First calibrate for the training phase — `optimizer` and `criterion` are required:

```python
from signai import monitor
import torch

optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
criterion = torch.nn.CrossEntropyLoss()

m = monitor.attach(model, num_classes=10)
m.calibrate(
    train_loader,
    device="cpu",
    phase="training",
    optimizer=optimizer,
    criterion=criterion,
)
m.save("./integrity_train.json")
```

Then score each training step after `optimizer.step()`:

```python
m = monitor.load(model, artifact="./integrity_train.json", device="cpu")

for x, y in train_loader:
    optimizer.zero_grad()
    logits = model(x)
    loss = criterion(logits, y)
    loss.backward()
    optimizer.step()

    result = m.score_training(logits, loss)
    if result.flagged:
        quarantine_update(result)
```

## Inference Loop Integration

```python
from signai import monitor

m = monitor.load(model, artifact="./integrity.json", device="cuda")

for x, y in test_loader:
    result = m.score_inference(x, y)
    if result.flagged:
        route_to_fallback(x)
```

## Public API

Top-level exports from `signai`:

| Symbol | Description |
|--------|-------------|
| `monitor.attach(model, num_classes, ...)` | Create a new monitor |
| `monitor.load(model, artifact, ...)` | Load from artifact or remote |
| `Monitor` | Monitor class with `calibrate`, `save`, `score_inference`, `score_training` |
| `MonitorResult` | Score result dataclass |
| `IntegrityMonitor` | Advanced: unified monitor with extractor + detector_kind params |
| `SignatureExtractorBase` | Advanced: ABC for custom extractor plugins |
| `ClassificationExtractor` | Classifier extractor (CNN, ViT, HF classification) |
| `LLMExtractor` | LLM extractor (generative transformers, >200M param models) |

## Run The Server

### CLI

```bash
signai-server serve \
  --host 0.0.0.0 \
  --port 8000 \
  --storage ./artifacts \
  --store-backend file
```

Shortcut through the main CLI:

```bash
signai serve --host 0.0.0.0 --port 8000 --storage ./artifacts
```

### Docker

```bash
docker build -t signai .
docker run -p 8000:8000 -v %cd%\artifacts:/data signai
```

### docker-compose

```bash
docker compose up --build
```

## Licensing

All usage requires a license key. A bundled trial key is active on fresh installs — no purchase needed to get started.

```bash
signai apply-key sk_...    # activate or renew a key
signai status              # show seat, plan, features, expiry, usage
```

Visit https://umarjanjua.github.io/signai/ to purchase or renew.

## API Summary

Server endpoints:

- `GET /health`
- `POST /v1/artifacts`
- `GET /v1/artifacts/{monitor_id}`
- `DELETE /v1/artifacts/{monitor_id}`
- `POST /v1/score/inference`
- `POST /v1/score/training`
- `POST /v1/score/batch`
- `POST /v1/calibrate/start`
- `POST /v1/calibrate/push`
- `POST /v1/calibrate/commit`
- `GET /v1/history/{monitor_id}`
- `POST /v1/notify/configure`
- `GET /v1/audit/export`
- `GET /v1/status`
- `GET /v1/monitors`
- `POST /v1/license`

## Model Support

| Framework | Status |
|-----------|--------|
| PyTorch (CNN, ViT, ResNet, etc.) | ✅ Supported |
| HuggingFace Transformers (BERT, GPT, LLaMA, etc.) | ✅ Supported |
| Custom PyTorch architectures via extractor plugin | ✅ Supported |
| TensorFlow / Keras | Planned |
| JAX / Flax | Planned |
| ONNX | Planned |
| XGBoost / LightGBM / CatBoost | Planned |
| PyG / DGL (graph neural networks) | Planned |

## Privacy Model

signAI is built so that privacy is enforced structurally:

- feature extraction runs on the customer machine
- remote scoring receives only `s` and `z` float vectors (compact; typically <100 bytes per score call)
- the server discards raw vectors after scoring
- only `{ts, score, flagged, phase}` is stored in history
- model weights, raw inputs, and gradients never leave the customer machine

## Documentation

- User manual: [USER_MANUAL.md](USER_MANUAL.md)
- Deployment guide: [DEPLOY.md](DEPLOY.md)
- Changelog: [CHANGELOG.md](CHANGELOG.md)
- Release checklist: [RELEASE_CHECKLIST.md](RELEASE_CHECKLIST.md)
- Product strategy, benchmarks, roadmap: [PRODUCT.md](PRODUCT.md)
- Test plan (5-layer): [TEST_PLAN.md](TEST_PLAN.md)
- Fulfillment ops runbook: [FULFILLMENT.md](FULFILLMENT.md)

## Development

Run the targeted test suite:

```bash
python -m pytest tests\test_imports.py tests\test_local_backend.py tests\test_monitor.py tests\test_server.py
```

Run the end-to-end and security test suite (isolated — does not touch `~/.signai`):

```bash
python scripts\test_e2e.py
```

## Licensing System

signAI uses Ed25519 asymmetric key signing. Keys are offline-verifiable — no network call on validation.

**Key format:** `sk_<base64url(json_payload)>.<base64url(ed25519_sig)>`

**Payload fields:** `seat_id, email, plan, features[], model_limit, history_days, issued_at, expires_at`

**Enforcement:** two checkpoints — `signai/licensing.py` (SDK client) and `signai_server/usage.py` (daemon). Both embed the public key as a hardcoded constant; the `SIGNAI_PUBLIC_KEY_B64` env var is intentionally ignored (P0 bypass fix).

**Clock integrity:** both checkpoints carry a monotonic date ratchet (`last_checked` / `last_validated`). The effective date is `max(today, last_recorded)` — setting the system clock backwards has no effect on trial or key expiry.

**Trial:** 3 days from first install, tracked in `~/.signai/install.json`.

## Key Generation

Keys are generated via GitHub Actions only — the private key never leaves the `keygen` environment:

```
Actions → Generate License Key → Run workflow
```

For local testing with an ephemeral keypair:

```bash
python scripts\keygen.py genkeypair
python scripts\keygen.py generate --seat alice@acme.com --email alice@acme.com --plan individual --duration-days 30
python scripts\keygen.py verify sk_...
```

`SIGNAI_PRIVATE_KEY_B64` must be set for `generate`. Never commit it.

## Fulfillment

Automated: `fulfill-orders.yml` polls Stripe every 5 minutes, generates keys, and emails customers.

Manual fallback: see [FULFILLMENT.md](FULFILLMENT.md).

Stripe setup (one-time):

```bash
set STRIPE_SECRET_KEY=sk_live_...
python scripts\stripe_setup.py
```

GitHub Secrets required in the `keygen` environment: `STRIPE_SECRET_KEY`, `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASSWORD`, `NOTIFY_EMAIL`.

## Release

Wheels are built via Cython + cibuildwheel on GitHub Actions — Python source is compiled into `.so`/`.pyd` binaries before publishing to PyPI. No source code ships in the wheel.

Tag a release: `git tag v0.x.y && git push origin v0.x.y`

The `publish.yml` workflow handles PyPI upload and Docker push to GHCR automatically.

## License

Commercial license required. The private key and source code are confidential — do not share outside the team.
