Metadata-Version: 2.5
Name: aegis-ai-router
Version: 1.0.0
Summary: Sub-6ms, zero-egress LLM routing, prompt-injection defense, and tool dispatch gateway.
Project-URL: Homepage, https://github.com/devxa2602/AegisRouter
Project-URL: Repository, https://github.com/devxa2602/AegisRouter
Project-URL: Bug Tracker, https://github.com/devxa2602/AegisRouter/issues
Author-email: Akshay Rawate <akshayrawate@gmail.com>
License: Apache-2.0
License-File: LICENSE
Keywords: fastapi,gateway,llm,onnx,prompt-injection,router
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: fastapi>=0.100.0
Requires-Dist: httpx>=0.24.0
Requires-Dist: huggingface-hub>=0.16.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: onnxruntime>=1.15.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0.0
Requires-Dist: transformers>=4.30.0
Requires-Dist: uvicorn[standard]>=0.20.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Provides-Extra: gpu
Requires-Dist: onnxruntime-gpu>=1.15.0; extra == 'gpu'
Description-Content-Type: text/markdown

# Aegis Router v1

**Sub-6ms, zero-egress LLM routing, prompt-injection defense, and tool dispatch in a single forward pass.**

[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
[![Hugging Face](https://img.shields.io/badge/HuggingFace-Weights-yellow.svg)](https://huggingface.co/AkshaXA/aegis-router-v1)
[![Live Demo](https://img.shields.io/badge/Demo-WebGPU_Zero_Egress-green.svg)]([https://devxa2602.github.io/AegisRouter/aegisRouter.html])

![Aegis Router T4 Benchmark GIF](assets/t4_demo.gif)
*NVIDIA T4 GPU (ONNX Runtime FP16, BS=1): 5.91 ms p50 | 157.3 req/sec*

---

## Try It Locally (Zero Data Egress)

Run the full ONNX FP16 model **100% inside your browser** via WebGPU/WASM—no signup, no backend, zero API calls:

👉 **[Launch Zero-Egress Browser Demo](https://devxa2602.github.io/AegisRouter/aegisRouter.html)**

*Open Chrome DevTools → Network tab after loading the ~300 MB weights to verify zero network requests leave your machine. Browser WebGPU runs at ~100 ms; T4 server latency is **5.91 ms p50**.*

---

## Why Aegis Router?

Production LLM gateways usually force a bad trade-off:
* **Rule-based proxies (LiteLLM/Kong):** Fast, but lack semantic understanding for model tiering or adversarial prompt detection.
* **Hosted routers:** Add 100–500 ms of network latency and send raw user prompts (and PII) to a third-party API.
* **LLM-as-a-Router (Haiku/GPT-4o-mini):** Burns tokens and adds 400–800 ms before your actual LLM call even starts.

**Aegis Router** runs inside your own VPC using a single 149M-parameter `ModernBERT-base` encoder pass (zero tokens generated) with three multi-task heads:
1. **Calibrated Model Tiering (`SLM` / `Mid` / `Frontier`):** Confidence threshold set at `0.85`—anything below automatically escalates to Frontier so routing errors fail safe.
2. **Prompt-Injection Guardrail:** Hard-blocks adversarial inputs before downstream LLM spend.
3. **5-Class Tool Dispatch:** Routes directly to `DIRECT_CHAT`, `CODE_OPS`, `RAG_RETRIEVAL`, `STRUCTURED_SQL`, or `EXTERNAL_API`.

---

## Benchmarks

Evaluated on an **NVIDIA T4 GPU** (ONNX Runtime FP16, `batch_size=1`):

| Benchmark | Aegis Router v1 |
| :--- | :--- |
| **p50 Latency** | **5.91 ms** |
| **p95 / p99 Latency** | 8.44 ms / 9.08 ms |
| **Throughput (BS=1)** | **157.3 req/sec** |
| **Prompt-Injection Guardrail** | **99.07% Acc** (0.9841 F1) |
| **5-Class Tool Dispatch** | **98.75% Acc** (0.9878 Macro-F1) |
| **Expected Calibration Error (ECE)** | **0.0572** |

![Aegis Router T4 Throughput Benchmark](assets/t4_benchmark.gif)
*Reproducible Kaggle T4 run (`N=500`, `BS=1`) — see [`notebooks/aegis_t4_benchmark.ipynb`](notebooks/aegis_t4_benchmark.ipynb)*

---

## Python Package & Gateway Quickstart

### 1. Install

```bash
# Editable install (for development)
pip install -e .

# Or install directly from GitHub
pip install git+https://github.com/devxa2602/AegisRouter.git

# Optional: GPU acceleration
pip install -e ".[gpu]"
```

### 2. Initialize Config

```bash
aegis-router init
```

This creates `aegis_config.yaml` in your current directory. Edit it to:

- **Swap models per tier** — point tier 0/1/2 at any OpenAI-compatible endpoint (vLLM, Ollama, Gemini, Bedrock gateways):
  ```yaml
  tiers:
    0:
      model: "my-local-llama"
      base_url: "http://localhost:8080/v1/chat/completions"
      api_key_env: "LOCAL_API_KEY"
  ```
- **Set custom pricing / zero-cost local models:**
  ```yaml
  pricing:
    overrides:
      my-local-llama:
        input_cost_per_1m: 0.0
        output_cost_per_1m: 0.0
  ```
- **Air-gapped VPC?** Set `pricing.offline_mode: true` — Aegis falls back to its local cache:
  ```yaml
  pricing:
    offline_mode: true
    cache_path: "~/.cache/aegis_router/model_prices.json"
  ```

### 3. Start the Gateway

```bash
aegis-router serve --config aegis_config.yaml --host 0.0.0.0 --port 8000
```

Then swap one line in your OpenAI SDK client:

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",   # ← point here
    api_key="your-api-key",
)

resp = client.chat.completions.create(
    model="ignored-by-aegis",  # Aegis routes to the right tier automatically
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(resp.choices[0].message.content)
```

### 4. ROI Reporting

**CLI:**
```bash
aegis-router report --days 7 --db aegis_telemetry.db
```

**API:**
```bash
curl http://localhost:8000/v1/roi?days=7
```

Both return: total requests, blocked injections, fail-safe escalations, p50/p95/p99 Aegis latency, actual spend vs. all-frontier counterfactual, net savings ($ and %), breakdown by tier & tool, and count of UNPRICED requests.

---

## Quickstart (Direct ONNX Inference)

### 1. Download Weights
Weights are hosted on Hugging Face: [AkshaXA/aegis-router-v1](https://huggingface.co/AkshaXA/aegis-router-v1)

```bash
pip install onnxruntime-gpu transformers huggingface_hub
```

### 2. Run Inference

```python
from huggingface_hub import hf_hub_download
import onnxruntime as ort
from transformers import AutoTokenizer
import numpy as np

model_path = hf_hub_download("AkshaXA/aegis-router-v1", "aegis_router_v1_fp16.onnx")
tokenizer = AutoTokenizer.from_pretrained("AkshaXA/aegis-router-v1")
session = ort.InferenceSession(model_path, providers=["CUDAExecutionProvider", "CPUExecutionProvider"])

inputs = tokenizer("Write a SQL query to find monthly active users.", return_tensors="np", truncation=True)
outputs = session.run(None, {
    "input_ids": inputs["input_ids"].astype(np.int64),
    "attention_mask": inputs["attention_mask"].astype(np.int64),
})
```

---

### Roadmap & Enterprise

Open-Source (Apache 2.0):
* **Aegis Router v1 weights (149M), ONNX FP16 graph, basic FastAPI proxy, and WebGPU demo**.
* **Coming in v2 (Aegis 300M)**: Custom 3:1 Mamba + FlashAttention-2 hybrid backbone for linear $O(N)$ routing and indirect prompt-injection defense across 64K–128K multi-turn agent trajectories.
* **Enterprise Control Plane**: Custom LoRA heads for proprietary tool schemas, RBAC, and SOC2 audit logging. Interested in a free in-VPC design partnership? Reach out at akshayrawate@gmail.com.

License: Apache 2.0 — see [LICENSE](LICENSE) for details.
