Metadata-Version: 2.4
Name: eyestech-mla
Version: 1.0.0
Summary: DeepSeek Multi-Head Latent Attention (MLA) reference kernel with absorbed query decoding and KV cache memory scaling benchmarks.
Author-email: Klaus Fischer <research@eyestech.in>, Devika Ranganathan <research@eyestech.in>
License-Expression: MIT
Project-URL: Homepage, https://eyestech.in/deepseek-mla-architecture-kv-cache-math/
Project-URL: Documentation, https://eyestech.in/deepseek-mla-architecture-kv-cache-math/
Project-URL: Repository, https://github.com/abhishek2512mishra/deepseek-mla-kvcache
Project-URL: Changelog, https://eyestech.in/
Keywords: deepseek,mla,multi-head-latent-attention,kv-cache,transformer,llm-inference,pytorch,attention,query-absorption,vllm,sglang
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.0
Dynamic: license-file

# eyestech-mla

[![PyPI Version](https://img.shields.io/pypi/v/eyestech-mla.svg?style=flat-square&color=blue)](https://pypi.org/project/eyestech-mla/)
[![EyesTech Systems Research](https://img.shields.io/badge/EyesTech-Systems_Research-002050?style=flat-square&logo=gitbook)](https://eyestech.in/deepseek-mla-architecture-kv-cache-math/)
[![Python Versions](https://img.shields.io/pypi/pyversions/eyestech-mla.svg?style=flat-square)](https://pypi.org/project/eyestech-mla/)
[![PyTorch 2.0+](https://img.shields.io/badge/PyTorch-2.0+-EE4C2C.svg?style=flat-square&logo=pytorch)](https://pytorch.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-emerald.svg?style=flat-square)](https://opensource.org/licenses/MIT)

**eyestech-mla** is a production-grade, standalone PyTorch reference kernel and memory scaling benchmark suite for **DeepSeek Multi-Head Latent Attention (MLA)**. It demonstrates **Query Absorption**, **Decoupled Rotary Position Embedding (RoPE)**, and **Zero-Decompression KV Cache Streaming** for high-throughput LLM inference up to 128k context windows.

> 📖 **Canonical Systems Audit & Mathematical Proof**:  
> For the complete mathematical proof, memory bandwidth derivations, and production serving economics (vLLM / SGLang), read the flagship investigation:  
> 👉 **[DeepSeek MLA Architecture: How It Cuts KV Cache by 93%](https://eyestech.in/deepseek-mla-architecture-kv-cache-math/)** published by **[EyesTech Systems Lab](https://eyestech.in)**.

---

## ⚡ Key Architectural Highlights

In autoregressive large language model serving, the primary hardware bottleneck at 32k–128k sequence lengths is **High Bandwidth Memory (HBM) exhaustion**, not raw compute FLOPs.

| Attention Mechanism | Stored Scalars / Token / Layer | KV Cache @ 128k Context (BS=8, FP16) | Hardware Feasibility Wall |
| :--- | :---: | :---: | :--- |
| **Frontier 128-Head MHA** | 32,768 | **3,840 GB** | ❌ Exceeds multi-node GPU clusters |
| **Standard 32-Head MHA** | 8,192 | **960 GB** | ❌ Requires 12x 80GB H100 GPUs |
| **Llama-3 8-Head GQA** | 2,048 | **240 GB** | ⚠️ Requires 3x 80GB H100 GPUs |
| **DeepSeek MLA (Absorbed)** | **576** | **67.5 GB** |  **Fits on a single 80GB H100 GPU (-93%)** |

### How DeepSeek MLA Achieves 92.97% Memory Compression:
1. **Low-Rank Latent Compression ($c_t^{KV}$)**: Projects multi-head Key and Value tensors into a compact 512-dimensional shared latent subspace ($d_c = 512$).
2. **Decoupled Rotary Position Embedding ($k_t^R$)**: Employs a dedicated 64-dimensional uncompressed RoPE key vector ($d_h^R = 64$), preserving exact positional distance semantics without inflating latent memory ($512 + 64 = 576$ scalars per token).
3. **Inference Query Absorption ($q_{\text{absorbed}} = q \cdot W_{UK}$)**: Leverages associative matrix multiplication during autoregressive decoding. The up-projection matrices are pre-folded into the active Query tensors, allowing attention dot-products directly against compressed latent states with zero decompression overhead.

---

## 🚀 Installation

Install via pip:

```bash
pip install eyestech-mla
```

Or install from source:

```bash
git clone https://github.com/abhishek2512mishra/deepseek-mla-kvcache.git
cd deepseek-mla-kvcache
pip install .
```

---

## 💻 Quickstart & Code Examples

### 1. Autoregressive MLA Decoding with Query Absorption

```python
import torch
from eyestech_mla import MultiHeadLatentAttentionDecode

device = "cuda" if torch.cuda.is_available() else "cpu"

# Instantiate DeepSeek MLA kernel matching V2/V3 configurations
mla = MultiHeadLatentAttentionDecode(
    d_model=5120,
    n_heads=128,
    d_head=128,
    d_latent=512,
    d_rope=64
).to(device)

# Simulate 128k context stream at position 1,024
batch_size = 2
seq_len = 1024
h_t = torch.randn(batch_size, 1, 5120, device=device)
cache_latent = torch.randn(batch_size, seq_len, 512, device=device)
cache_rope = torch.randn(batch_size, seq_len, 64, device=device)

# Execute single-token decode step
output, new_latent, new_rope = mla.forward_decode(
    h_t,
    current_pos=seq_len,
    kv_cache_latent=cache_latent,
    kv_cache_rope=cache_rope
)

print(f"Token Output Tensor: {output.shape}")          # [2, 1, 5120]
print(f"Updated Latent Cache: {new_latent.shape}")      # [2, 1025, 512]
print(f"Updated RoPE Cache:   {new_rope.shape}")        # [2, 1025, 64]
print(f"Stored Scalars/Token: {new_latent.shape[-1] + new_rope.shape[-1]}")  # 576
```

### 2. KV Cache Scaling Benchmark (CLI)

Run the automated scaling benchmark directly from your terminal:

```bash
eyestech-mla-benchmark
```

### 3. Programmatic Memory Benchmark API

```python
from eyestech_mla import compute_kv_cache_bytes, format_bytes

# Compute KV cache for 128k context, 60 layers, batch size 8 in FP16
bytes_mla = compute_kv_cache_bytes(
    n_layers=60,
    n_kv_heads=128,
    d_head=128,
    seq_len=131072,
    batch_size=8,
    bytes_per_elem=2,
    is_mla=True,
    d_latent=512,
    d_rope=64
)

bytes_mha = compute_kv_cache_bytes(
    n_layers=60,
    n_kv_heads=128,
    d_head=128,
    seq_len=131072,
    batch_size=8,
    bytes_per_elem=2,
    is_mla=False
)

print(f"DeepSeek MLA Footprint: {format_bytes(bytes_mla)}")  # 67.50 GB
print(f"Frontier MHA Footprint: {format_bytes(bytes_mha)}")  # 3840.00 GB
```

---

## 📊 Benchmark Telemetry (60 Layers, Batch Size = 8, FP16)

```text
========================================================================================
Architecture               | Scalars/Tok | 4k (4096)  | 16k (16384) | 32k (32768) | 64k (65536) | 128k (131072)
----------------------------------------------------------------------------------------
Standard MHA (32-head)     | 8192        | 30.00 GB   | 120.00 GB   | 240.00 GB   | 480.00 GB   | 960.00 GB   
Frontier MHA (128-head)    | 32768       | 120.00 GB  | 480.00 GB   | 960.00 GB   | 1920.00 GB  | 3840.00 GB  
Llama-3 GQA (8-head)       | 2048        | 7.50 GB    | 30.00 GB    | 60.00 GB    | 120.00 GB   | 240.00 GB   
DeepSeek MLA (Absorbed)    | 576         | 2.11 GB    | 8.44 GB     | 16.88 GB    | 33.75 GB    | 67.50 GB    
========================================================================================
```

---

## 📚 Citation & Attribution

If you use this benchmark harness, reference implementation, or mathematical formalization in academic research, benchmark audits, or engineering systems, please cite:

```bibtex
@misc{fischer2026deepseekmla,
  author = {Fischer, Klaus and Ranganathan, Devika},
  title = {DeepSeek MLA Architecture: How It Cuts KV Cache by 93%},
  howpublished = {\url{https://eyestech.in/deepseek-mla-architecture-kv-cache-math/}},
  journal = {EyesTech Systems Research},
  year = {2026},
  note = {EyesTech Systems Lab Hardware Audit Series}
}
```

---

## ⚖️ License

Distributed under the [MIT License](https://opensource.org/licenses/MIT). Maintained by [EyesTech Systems Lab](https://eyestech.in).
