Metadata-Version: 2.4
Name: transformers-v9
Version: 0.1.0
Summary: V9 Palace-structured attention for Transformers — block-diagonal palace-collated reasoning, per-palace LoRA, cross-palace gating
Author-email: Wayne <wayne@siliconlifeos.ai>
License: MIT
Project-URL: Homepage, https://siliconlifeos.ai
Project-URL: Repository, https://github.com/anomalyco/SiliconLifeOS
Keywords: transformer,attention,palace,spc,v9,llm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: torch>=2.0
Requires-Dist: transformers>=4.30
Requires-Dist: accelerate>=0.20

# transformers-v9

V9 Palace-structured attention for HuggingFace Transformers — block-diagonal palace-collated reasoning, per-palace LoRA adapters, cross-palace gating.

## Install

```bash
pip install transformers-v9
```

## Quick Start

```python
import transformers_v9  # registers v9_palace with Auto Classes

from transformers import AutoConfig, AutoModelForCausalLM

config = AutoConfig.from_pretrained("v9_palace", vocab_size=32000, hidden_size=4096)
model = AutoModelForCausalLM.from_pretrained(config)
```

## Architecture

- **PalaceAttentionLayer**: Low-rank QKV (r=64) + Palace block-diagonal mask + per-palace LoRA K/V (r=9, 38 palaces) + cross-palace gating
- **6 residual layers** (stacked on frozen base LLM), no FFN — base model provides it
- **~50.4M trainable params** at hidden_size=4096

## Training

```python
from transformers_v9 import V9PalaceTrainer

trainer = V9PalaceTrainer(model, train_dataset, eval_dataset)
trainer.train()
```

## Requirements

- Python 3.8+
- PyTorch 2.0+
- transformers 4.30+
- accelerate 0.20+
