Metadata-Version: 2.4
Name: transformer-lib
Version: 0.1.0
Summary: Attention mechanisms from Every_Thing_About_Transformer, as an importable library
Author: Aniket Verma
License: MIT
Requires-Python: >=3.9
Requires-Dist: torch>=2.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == 'dev'
Provides-Extra: linear
Requires-Dist: linear-attention-transformer; extra == 'linear'
Description-Content-Type: text/markdown

# transformer-lib

Attention mechanisms from Every_Thing_About_Transformer, as an importable Python library.

## Installation

```bash
pip install -e .
```

## Usage

```python
from transformer_lib import MultiHeadAttention
import torch

mha = MultiHeadAttention(embed_dim=512, num_heads=8)
x = torch.randn(2, 10, 512)
out = mha(x, x, x)
```

## Available Modules

| Module | Description |
|---|---|
| `SelfAttention` | Basic scaled dot-product self-attention |
| `MultiHeadAttention` | Multi-head attention with separate Q/K/V projections |
| `CausalAttention` | Multi-head with lower-triangular causal mask |
| `CrossAttention` | Encoder-decoder cross-attention with self-attention pre-step |
| `GlobalAttention` | Additive (Bahdanau-style) alignment scoring |
| `MultiQueryAttention` | Shared K/V across all heads |
| `GroupedQueryAttention` | Groups of query heads sharing K/V (GQA) |
| `MultiHeadLatentAttention` | Latent-space compression for keys/values |
