Metadata-Version: 2.4
Name: transformers-v9
Version: 0.2.0
Summary: V9 Palace Transformer — from-scratch LLM with block-diagonal palace-collated attention, per-palace LoRA K/V, cross-palace gating, and semantic palace routing
Author-email: Wayne <wayne@siliconlifeos.ai>
License: MIT
Project-URL: Homepage, https://siliconlifeos.ai
Project-URL: Repository, https://github.com/anomalyco/SiliconLifeOS
Keywords: transformer,attention,palace,spc,v9,llm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: torch>=2.0
Requires-Dist: transformers>=4.30
Requires-Dist: accelerate>=0.20

# transformers-v9

**V9 Palace Transformer** — an independent from-scratch LLM architecture with block-diagonal palace-collated attention, per-palace LoRA K/V adapters, and cross-palace semantic gating. Not a base-model adapter — its own embedding/position/lm_head stack.

Part of the V∞ (SiliconLifeOS) paradigm: cognitive structure is architecture, not prompt engineering.

## Architecture

| Component | Detail |
|-----------|--------|
| Palace Attention | Block-diagonal mask: tokens attend only within same palace |
| LoRA K/V | Per-palace low-rank adapters (r=16, 48 palaces) |
| Cross-palace gate | Softmax MLP mix of palace embeddings |
| QKV low-rank | D→64→D bottleneck projections |
| Residuals | 6 layers, no FFN (sufficient for structured reasoning) |
| Default scale | ~1.5B params (hidden=2048, 6 layers, 48 palaces) |

```
Input → Embed/Pos → 6×PalaceAttentionLayer (residual) → LM Head → Output
```

## Quick Start

```python
import transformers_v9  # registers v9_palace with HF Auto Classes

from transformers import AutoConfig, AutoModelForCausalLM

config = AutoConfig.from_pretrained("v9_palace", vocab_size=32768, hidden_size=2048)
model = AutoModelForCausalLM.from_config(config)
```

## Training

```python
from transformers_v9 import V9PalaceTrainer

trainer = V9PalaceTrainer(model, train_dataset, eval_dataset)
trainer.train()
```

From-scratch training pipeline included (`from transformers_v9.train import get_curriculum_dataset`):
- SPC L1-L6 curriculum data
- CP fusion tasks
- Multi-domain reasoning

## Router (Semantic Palace Routing)

```python
from transformers_v9 import V9RouterConfig, V9Router

router = V9Router(V9RouterConfig())
palace = router.route("Discuss the attention mechanism in Transformers")
# → Palace ID for attention-related reasoning
```

4-level cascade: L0 domain hash → L1 keyword voting → L2 group Jaccard → L3 embedding (optional).

## Install

```bash
pip install transformers-v9
```

## Requirements

- Python 3.8+
- PyTorch 2.0+
- transformers 4.30+
- accelerate 0.20+

## Status

Alpha prototype — structured reasoning validation complete. CPU training verified (500 samples, 20 epochs: ont_self 0.55→1.11, accuracy 96.5%). GPU training for full-scale (1.5B+) recommended.

## Links

- [V∞ Paradigm](https://siliconlifeos.ai)
- [GitHub](https://github.com/anomalyco/SiliconLifeOS)
