Metadata-Version: 2.4
Name: zesyde
Version: 1.0.0
Summary: Calibrated, Non-Autoregressive Decision & Routing Foundation Models
Author: Zoeb Chhatriwala
License: MIT
Project-URL: Homepage, https://github.com/zoebchhatriwala/Zesyde
Keywords: nlp,routing,onnx,decision-making,llm-router
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: torch
Requires-Dist: transformers
Requires-Dist: datasets
Requires-Dist: accelerate
Requires-Dist: scikit-learn
Requires-Dist: scipy
Requires-Dist: pydantic>=2.0
Requires-Dist: numpy
Requires-Dist: tqdm
Requires-Dist: onnx
Requires-Dist: onnxruntime
Requires-Dist: onnxscript
Requires-Dist: fastapi
Requires-Dist: uvicorn
Requires-Dist: peft
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"

---
language:
- multilingual
- en
- es
- hi
- ar
- fr
- de
- zh
- tr
- vi
- ru
license: mit
library_name: transformers
tags:
- decision-model
- classification
- non-autoregressive
- calibrated
- calibration
- onnx
- zero-shot
- routing
- agent-router
datasets:
- facebook/belebele
- xnli
metrics:
- accuracy
- brier_score
pipeline_tag: text-classification
---

# Zesyde: Calibrated, Non-Autoregressive Decision & Routing Foundation Model

[![Version](https://img.shields.io/badge/version-1.0.0-blue.svg)](pyproject.toml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Latency](https://img.shields.io/badge/Latency-12--35ms_CPU-brightgreen.svg)]()
[![Model Size](https://img.shields.io/badge/ONNX_INT8-318MB-success.svg)]()
[![Parameters](https://img.shields.io/badge/Parameters-313M_(150M_Core)-blue.svg)]()
[![Context](https://img.shields.io/badge/Context-8192_Tokens-orange.svg)]()

**Zesyde** is a production-ready, non-autoregressive foundation model engineered for **ultra-fast, mathematically calibrated zero-shot decisions and agent routing** across 1,800+ languages.

Instead of generating text token-by-token with massive generative LLMs, Zesyde scores arbitrary candidate sets simultaneously in **12–35 milliseconds on pure CPU** using candidate-token pooling, prompt-context fusion, and out-of-sample temperature scaling.

---

## 🚀 Key Highlights

* **Blazing Fast (Non-Autoregressive)**: Generates decisions in **12–35ms** on pure CPU via INT8 ONNX Runtime (p50 ~12ms on short sequences / AVX-512, ~33ms average across benchmarks). No autoregressive token generation overhead.
* **Mathematically Calibrated**: Built-in post-hoc temperature scaling fitted via strictly proper scoring rules. Minimizes Expected Calibration Error (**ECE < 2.5%**). Predicted confidence matches empirical accuracy.
* **Massive Cross-Lingual Coverage**: Native support across **1,800+ languages** utilizing `jhu-clsp/mmBERT-base` with zero-dependency heuristic script routing. 313M total parameters (150M-class ModernBERT transformer core + 256k-token multilingual embedding table).
* **8,192-Token Architectural Context**: RoPE-based rotary positional embeddings and dynamic ONNX axes support sequences up to **8,192 tokens** (fine-tuned up to 1,024 tokens).
* **Autonomous Escalation Head (`Act/Escalate`)**: Built-in binary routing logit that autonomously flags low-confidence or high-risk inputs for human review or fallback to heavier frontier models.
* **Drop-in CPU Serving**: Optimized FP32 and INT8 ONNX graphs with dynamic quantization and a standalone FastAPI + aesthetic web dashboard.

---

## 📊 Benchmark & Evaluation Results

Evaluated on held-out test splits after full 150K multi-task multilingual training with Regularized Log-Loss Calibration Distillation (RLCD):

| Benchmark / Task | Language & Primitive | Accuracy | Expected Calibration Error (ECE) | Brier Score |
| :--- | :--- | :---: | :---: | :---: |
| **`belebele_es`** | Spanish (4-way QA) | **`89.0%`** | **`0.0447`** *(4.5%)* | **`0.1608`** |
| **`belebele_ar`** | Arabic (4-way QA) | **`83.5%`** | **`0.0342`** *(3.4%)* | **`0.2362`** |
| **`belebele_hi`** | Hindi (4-way QA) | **`73.5%`** | **`0.0786`** *(7.9%)* | **`0.3452`** |
| **`xnli_de`** | German (Boolean NLI) | **`87.0%`** | **`0.0577`** *(5.8%)* | **`0.2153`** |
| **`xnli_fr`** | French (3-way NLI) | **`79.0%`** | **`0.0881`** *(8.8%)* | **`0.3259`** |
| **OVERALL AGGREGATE** | **Cross-Lingual Held-Out** | **`82.4%`** | **`0.0188`** *(1.9%)* | **`0.2567`** |

### Verified Post-Hoc Calibration Temperatures
Fitted via strictly proper scoring rules on independent held-out calibration partitions:
* $T[\text{choice} \mid K=4] = \mathbf{0.8528}$
* $T[\text{choice} \mid K=3] = \mathbf{1.8290}$
* $T[\text{noul} \mid K=2] = \mathbf{1.0408}$
* $T[\text{default}] = \mathbf{1.0000}$

*All temperature scaling factors are closely grouped around 1.0, proving natural calibration without logit explosion or collapse.*

---

## ⚡ Quickstart

### Installation

```bash
pip install -e .
```

### 1. Python SDK (3-Line Inference)

```python
from zesyde.router import ZesydeRouter

# Load calibrated INT8 ONNX engine (or omit prefer_fp32 for FP32)
router = ZesydeRouter.from_pretrained("models", prefer_fp32=False, max_length=1024)

# Make a calibrated, fast routing decision
result = router.decide(
    context="The new flagship battery lasts for over 3 days on a single charge.",
    question="What is the sentiment of this review?",
    options=["Negative", "Neutral", "Positive"]
)

print(f"Selected: {result.selected_option}")       # "Positive"
print(f"Confidence: {result.confidence:.3f}")        # 0.982
print(f"Should Escalate: {result.should_escalate}")  # False
print(f"Latency: {result.latency_ms:.1f}ms")         # ~12.4ms (short sequence)
```

---

### 2. Autonomous Escalation & Risk Routing

Zesyde includes a dedicated `Act/Escalate` decision head that triggers when input ambiguity is high:

```python
decision = router.decide(
    context="Ambiguous text or high-risk medical prompt...",
    question="Determine action plan:",
    options=["Auto-Approve", "Reject"]
)

if decision.should_escalate:
    # Route to human review queue or heavier reasoning LLM (e.g. Gemini / Claude)
    escalate_to_human_reviewer(decision)
else:
    # Safely execute automated decision
    execute_action(decision.selected_option)
```

---

### 3. REST API & Web Dashboard

Launch the FastAPI production server:

```bash
python scripts/api.py --model-dir models --port 8000
```

* **Interactive Web Dashboard**: Navigate to `http://localhost:8000/` for a dark-mode, real-time testing playground.
* **REST Endpoint**:

```bash
curl -X POST http://localhost:8000/decide \
  -H 'Content-Type: application/json' \
  -d '{
    "context": "El nuevo smartphone es increíble, la batería dura tres días seguidos.",
    "question": "What is the sentiment?",
    "options": ["Negative", "Neutral", "Positive"]
  }'
```

---

## 🏗️ Model Architecture

```text
Sequence Layout:
[CLS] context [SEP] question [SEP] [MASK] Option 1 [MASK] Option 2 ... [MASK] Option K [SEP]
                                      │                  │                   │
                                      ▼                  ▼                   ▼
                          Candidate Mean-Pooling  Candidate Mean-Pooling  Candidate Mean-Pooling
                                      │                  │                   │
                                      └──────────────────┼───────────────────┘
                                                         ▼
                                                Context Fusion Layer
                                          [h_opt, h_cls, h_opt ⊙ h_cls]
                                                         ▼
                                            Calibrated Decision Head
                                                         ▼
                                         Temperature Scaling Matrix T[Type|K]
                                                         ▼
                                          Calibrated Probability Vector
```

1. **Backbone**: Pretrained `jhu-clsp/mmBERT-base` (multilingual) or `answerdotai/ModernBERT-large` (English).
2. **Context Fusion**: Candidate option representations $\mathbf{h}_{\text{opt}}$ are concatenated with the sequence representation $\mathbf{h}_{\text{cls}}$ and their interaction $\mathbf{h}_{\text{opt}} \odot \mathbf{h}_{\text{cls}}$, yielding a $3H$-dimensional feature vector per option.
3. **Decision Head**: Regularized multi-layer perceptron with `GELU` and `LayerNorm` predicting unconstrained scalar compatibility logits.
4. **Escalation Head**: Binary classifier over $\mathbf{h}_{\text{cls}}$ predicting escalation probability.
5. **Temperature Scaling**: Post-hoc isotonic temperature table $T[\text{type} \mid K]$ learned on held-out calibration partitions.

---

## 🛠️ Reproduction & Training Pipeline

To reproduce the 128K production build end-to-end (supervised warm-start, RLCD distillation, temperature calibration, and INT8 ONNX export):

```bash
python scripts/build_prod_model.py \
  --multilingual \
  --train-limit 128000 \
  --epochs 1 \
  --batch-size 16 \
  --max-length 1024
```

### Mid-Training Pause & Resumption
Training can be paused safely at any point:
* Press **`p`** in the terminal to pause, save an in-memory or disk checkpoint, and quit.
* Press **`Ctrl+C`** for graceful signal capture to save `models/checkpoints/train_checkpoint.pt`.
* Resume training seamlessly with:
  ```bash
  python scripts/build_prod_model.py --resume
  ```

---

## 📦 Export Artifacts

Running the pipeline outputs complete serving artifacts to `models/`:
* `zesyde_prod_int8.onnx`: Dynamically quantized 8-bit ONNX model for CPU inference (~318 MB / ~95 MB non-embedding weights).
* `zesyde_prod_fp32.onnx`: Full-precision 32-bit ONNX model (~1.16 GB).
* `zesyde_calibration.json`: Persisted temperature scaling lookup table.
* `zesyde_trained.pt`: Full PyTorch model checkpoint (~1.17 GB, 313.1M parameters).

---

## 📜 Citation & License

```bibtex
@software{zesyde2026,
  author = {Chhatriwala, Zoeb},
  title = {Zesyde: Calibrated, Non-Autoregressive Decision & Routing Foundation Models},
  year = {2026},
  version = {1.0.0},
  url = {https://huggingface.co/zoebchhatriwala/Zesyde-150M-Multilingual-Router}
}
```

Licensed under the [MIT License](LICENSE).
