Metadata-Version: 2.5
Name: saccade-ai
Version: 0.1.1
Summary: Intrinsic Attention Bottlenecks for Vision Transformers & LLM Prompt Compression.
Project-URL: Homepage, https://github.com/helloitsdaksh/saccade
Project-URL: Repository, https://github.com/helloitsdaksh/saccade
Project-URL: Documentation, https://github.com/helloitsdaksh/saccade#readme
Author-email: Daksh Patel <helloitsdaksh007@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: attention-bottleneck,explainable-ai,interpretability,llm,mechanistic-interpretability,prompt-compression,pytorch,token-pruning,vision-transformer
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Requires-Dist: matplotlib>=3.5.0
Requires-Dist: numpy>=1.20.0
Requires-Dist: pillow>=9.0.0
Requires-Dist: scipy>=1.9.0
Requires-Dist: timm>=0.9.0
Requires-Dist: torch>=2.0.0
Requires-Dist: torchvision>=0.15.0
Requires-Dist: transformers>=4.30.0
Provides-Extra: dev
Requires-Dist: build>=1.0.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Requires-Dist: twine>=4.0.0; extra == 'dev'
Description-Content-Type: text/markdown

# saccade 👁️⚡

[![PyPI version](https://img.shields.io/pypi/v/saccade-ai.svg?color=blue)](https://pypi.org/project/saccade-ai/)
[![GitHub Repo](https://img.shields.io/badge/GitHub-helloitsdaksh%2Fsaccade-181717.svg?logo=github)](https://github.com/helloitsdaksh/saccade)
[![Live Demo](https://img.shields.io/badge/Live%20Demo-Interactive%20Workbench-6366f1.svg)](https://dakshpatel.dev/demos/saccade-ai.html)
[![Python 3.9+](https://img.shields.io/badge/python-3.9+-brightgreen.svg)](https://www.python.org/)
[![PyTorch 2.0+](https://img.shields.io/badge/PyTorch-2.0+-ee4c2c.svg)](https://pytorch.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Causal Faithfulness](https://img.shields.io/badge/Causal%20Faithfulness-100%25%20Guaranteed-00ffcc.svg)](#guaranteed-causal-faithfulness)

> *"Don't explain the black box after the crime was committed. Physically blindfold the model."*

**saccade** is an open-source Python library for **Intrinsically Interpretable Vision Transformers (ViTs)** and **LLM Prompt Token Compression**. Instead of relying on post-hoc saliency heuristics (like Grad-CAM) that generate plausible-looking "interpretability illusions", `saccade` introduces an explicit **hard attention bottleneck** that physically discards uninformative tokens.

- 📖 **Deep Dive Essay:** [I Deleted 94% of an Image and 70% of an LLM Prompt](https://dakshpatel.dev/blog/i-deleted-94-of-an-image-and-70-of-an-llm-prompt.html)
- 🎮 **Live Interactive Demo:** [Scrub ViT Layers & LLM Tokens in Browser](https://dakshpatel.dev/demos/saccade-ai.html)
- 📦 **PyPI Package:** [`pip install saccade-ai`](https://pypi.org/project/saccade-ai/)

1. **Vision ViT Saccading**: Discard up to **94% of image patches** while preserving full diagnostic accuracy. Downstream predictions are causally bound only to the preserved $K$ coordinates ($\partial \hat{y} / \partial x_m \equiv 0$).
2. **LLM Prompt Saccading**: Prune **60% to 75% of input prompt tokens** by discarding syntactic fluff and conversational padding ("breaking grammar"), slashing API costs, prefill latency, and KV-cache footprint while maintaining reasoning fidelity.

---

## ⚡ Quickstart

Install via pip:

```bash
pip install saccade-ai
```

Or clone for local development:

```bash
git clone https://github.com/helloitsdaksh/saccade.git
cd saccade
pip install -e .
```

### 1. Vision ViT Saccade (3 Lines)

```python
import torch
from saccade import SaccadeViT, export_interactive_html

# 1. Instantiate ViT with intrinsic attention bottleneck
# Only 12 out of 196 patches (6.1% budget) reach downstream layers
model = SaccadeViT.from_pretrained(
    "vit_tiny_patch16_224",
    num_classes=1000,
    top_k_patches=12,
    bottleneck_layer=6
)

# 2. Forward pass returns class predictions and exact spatial metadata
image = torch.randn(1, 3, 224, 224)
logits, meta = model(image, return_saccades=True)

print("Preserved Patch Indices:", meta.patch_indices)
print("Bounding Boxes [ymin, xmin, ymax, xmax]:", meta.bounding_boxes)

# 3. Export standalone interactive web visualizer (zero dependencies)
export_interactive_html(model=model, image=image, output_path="saccade_viewer.html")
```

### 2. LLM Prompt Token Saccade (3 Lines)

```python
from saccade import PromptSaccade

# 1. Instantiate the prompt compressor
compressor = PromptSaccade(model_name="distilgpt2", device="cpu")

# 2. Compress prompt by 70% (strips syntactic fluff, retains semantic hubs)
prompt = (
    "Please carefully review and analyze the following quarterly report and "
    "summarize whether enterprise cloud software revenue grew year-over-year: "
    "'The company reported enterprise cloud revenue surged 34% to $6.8 billion.' "
    "What was the growth rate?"
)

result = compressor.compress(prompt, compression_ratio=0.30)

print(result.compressed_text)
# => "quarterly report summarize enterprise cloud software revenue grew: enterprise cloud revenue surged 34% $6.8 billion. What growth rate?"

print(f"Token reduction: {result.token_savings_pct:.1f}% ({result.original_tokens} -> {result.compressed_tokens} tokens)")
```

---

## 🔬 Core Mechanics: ViT Spatial Bottleneck

In standard ViTs, downstream classification heads observe all $N$ tokens across the full image sequence. `saccade` intercepts the self-attention affinity matrix $A$ at an intermediate probing layer $L_b$:

```
Input Image [B, C, H, W]
       │
       ▼
Patch Tokenizer & ViT Blocks 0 .. L_b [B, N, D]
       │
       ▼
Self-Attention Matrix A ∈ R^{N × N}  (Layer L_b)
       │
       ▼
Patch Importance Voting: p_j = ∑_{i=1}^N A_{i, j}
       │
       ▼
Hard Top-K Bottleneck (Discard all but K tokens)
       │
       ▼
Sparse Downstream Predictor Blocks (L_b + 1 .. L_end)
       │
       ▼
Class Prediction ŷ (Causally derived ONLY from K patches)
```

### 1. Patch Importance Voting ($p$)
For an input sequence of $N$ spatial patches, let $A \in \mathbb{R}^{N \times N}$ denote the attention affinity matrix from probing layer $L_b$. The **column sum** represents the total "votes" cast for patch $j$ by all other spatial querying locations:

$$p_j = \sum_{i=1}^N A_{i, j}$$

### 2. Hard Top-$K$ Spatial Saccade
The model sorts $p$ and routes only the top-$K$ patches (e.g., $K=12$ out of $196$ patches, representing $< 7\%$ of the visual scene) into subsequent transformer layers:

$$\mathcal{J}_K = \text{TopK}(p, K)$$

$$\mathbf{Z}_{\text{downstream}} = [\mathbf{z}_{\text{cls}}, \; \mathbf{z}_{\mathcal{J}_K}]$$

### 3. Guaranteed Causal Faithfulness
Downstream predictions are causally bound to only those $K$ patches. All non-selected patches are physically removed from the compute graph:

$$\frac{\partial \hat{y}}{\partial x_m} \equiv 0 \quad \forall m \notin \mathcal{J}_K$$

---

## 💬 LLM Prompt Token Saccading ("Breaking Grammar")

Large Language Models do not read like humans. Conversational politeness (*"Could you please assist me by..."*, *"It should be noted that in accordance with..."*) consumes up to **70% of prompt token budgets** and bloats the KV cache with low-information syntactic glue.

`PromptSaccade` runs a fast single-pass attention probe over the prompt to calculate:
1. **Self-Attention In-Degree**: Identifies semantic anchor tokens that subsequent clauses heavily attend to.
2. **Shannon Entropy Sieve**: Filters out predictable, high-probability syntactic padding.

### Empirical Prompt Compression Benchmark

Run the benchmark locally:
```bash
python examples/llm_prompt_saccade.py
```

| Task Category | Original Tokens | Saccaded Tokens | Tokens Saved | Token Reduction | Estimated Prefill Speedup |
| :--- | :---: | :---: | :---: | :---: | :---: |
| **Pathology / Medical Report** | 124 | 39 | 85 | **68.5%** | ~3.2× |
| **Financial SEC 10-Q Analysis** | 116 | 34 | 82 | **70.7%** | ~3.4× |
| **Python Bug Diagnostic** | 91 | 30 | 61 | **67.0%** | ~3.0× |
| **Average Across Workloads** | **110** | **34** | **76** | **68.9%** | **~3.2×** |

*Estimated API savings: **\$2,280 saved per 1M queries** (at \$10/M tokens).*

---

## 🌐 Standalone Interactive Web Artifacts

`saccade` includes a built-in exporter that compiles interactive scientific visualizers into a **single, zero-dependency, self-contained HTML file** with embedded base64 assets and client-side canvas rendering:

> 🚀 **Live Interactive Demo**: [dakshpatel.dev/demos/saccade-ai.html](https://dakshpatel.dev/demos/saccade-ai.html) — scrub ViT probing layers ($L_0$ to $L_{11}$), adjust token budgets ($K$), and inspect LLM prompt compression live in your browser.

- 🎚️ **Real-Time Sparsity Slider**: Vary $K$ dynamically from $1$ to $N$, observing how confidence shifts as visual context is revealed.
- 🎯 **Interactive Patch Ablation**: Click individual active patches to manually disable them, instantly viewing the counterfactual logit penalty:
  $$\Delta \text{logit} = \text{logit}_{\text{base}} - \text{logit}_{\text{ablated}}$$
- 👁️ **Scanpath Arcs**: Visualizes the model's saccadic fixation order with sleek vector lines and rank tags (`#1`, `#2`, ...).
- 🚀 **Zero Backend Required**: Pure client-side HTML5 Canvas + vanilla JS. No node servers, no Python backends, instant sharing on GitHub Pages or static hosts.

---

## 📊 Faithfulness Benchmark: Intrinsic vs Post-Hoc

| Property | Post-Hoc Saliency (Grad-CAM / IG) | Saccade Intrinsic Bottleneck |
| :--- | :---: | :---: |
| **Causal Bound** | ❌ Heuristic approximation | ✅ **100% Causal Guarantee** |
| **Gradient Leakage on Discarded Pixels** | ⚠️ Non-zero everywhere ($\|\nabla_x\| > 0$) | ✅ **Identically 0.000000** |
| **Pruning Capability** | ❌ None (observes full image) | ✅ **Prunes 94% of input tokens** |
| **AUC-Deletion (Steeper drop is better)** | ~0.35 | **0.166** |
| **Interactive Perturbation in Browser** | ❌ Requires live Python backend | ✅ **Zero-dependency standalone HTML** |

Run the quantitative benchmark:
```bash
python examples/benchmark_faithfulness.py
```

---

## 📁 Repository Structure

```
saccade/
├── saccade/
│   ├── __init__.py               # Public API exports
│   ├── bottleneck.py             # AttentionBottleneck & column-sum voting
│   ├── models.py                 # SaccadeViT wrapper & CleanViT native
│   ├── visualization.py          # Overlays, scanpaths, & matplotlib breakdown
│   ├── exporter.py               # Standalone interactive HTML generator
│   ├── benchmarks/
│   │   └── faithfulness.py       # AUC-Del, AUC-Ins, & gradient leakage test
│   └── text/
│       └── token_saccade.py      # LLM prompt token saccader & entropy compression
├── examples/
│   ├── quickstart.py             # ViT end-to-end quickstart & HTML exporter
│   ├── llm_prompt_saccade.py     # LLM prompt token pruning & cost benchmarks
│   ├── generate_layer_by_layer_inspector.py # Interactive layer visualizer generator
│   └── benchmark_faithfulness.py # Deletion & Insertion curve benchmarks
├── tests/
│   ├── test_bottleneck.py        # Bottleneck & voting unit tests
│   ├── test_models.py            # SaccadeViT & CleanViT tests
│   ├── test_exporter.py          # HTML export verification tests
│   ├── test_faithfulness.py      # Causal gradient isolation verification
│   └── test_text_saccade.py      # LLM prompt token saccading tests
├── assets/
│   └── samples/                  # Real evaluation scenes (Eagle, Panda, etc.)
├── CITATION.cff                  # Citation metadata for GitHub 1-click cite
├── LICENSE                       # MIT License
└── pyproject.toml                # Modern PEP 621 packaging
```

---

## 🧪 Running Tests

```bash
pytest tests/ -v
```

All 15 test suites verify:
- Column-sum patch importance voting correctness
- Normalized bounding box conversions
- Dynamic top-$K$ budget switching
- Standalone HTML generation with embedded base64 data
- Exact 0.0 causal gradient isolation
- Text token saccading & Shannon entropy compression

---

## 🙏 Credits & Acknowledgements

- **Inspiration:** The intrinsic attention column-voting mechanism was inspired by **David Ha** ([@hardmaru](https://twitter.com/hardmaru)) and the foundational paper *Neuroevolution of Self-Interpretable Agents* (Tang, Nguyen, & Ha, 2020).
- **Vision Backbones:** Built with Ross Wightman's [timm](https://github.com/huggingface/pytorch-image-models) library (`vit_small_patch16_224.dino`, `deit_small_patch16_224`).
- **Language Models:** DistilGPT2 / Hugging Face Transformers for causal attention extraction and next-token perplexity entropy.

---

## 📜 Citation

If you use `saccade` in your research, experiments, or applications, please cite:

```bibtex
@misc{patel2026saccade,
  author       = {Patel, Daksh},
  title        = {saccade: Intrinsic Attention Bottlenecks for Vision Transformers and LLM Prompt Compression},
  year         = {2026},
  month        = {November},
  howpublished = {\url{https://github.com/helloitsdaksh/saccade}},
  note         = {Version 0.1.1}
}
```

Or using BibLaTeX:

```bibtex
@software{patel2026saccade,
  author    = {Patel, Daksh},
  title     = {saccade: Intrinsic Attention Bottlenecks for Vision Transformers and LLM Prompt Compression},
  year      = {2026},
  month     = {11},
  version   = {0.1.1},
  publisher = {GitHub},
  url       = {https://github.com/helloitsdaksh/saccade}
}
```

## 📄 License
MIT License. Free for academic research, open-source development, and commercial use.
