Metadata-Version: 2.4
Name: spiral-lm
Version: 0.1.1
Summary: Spiral — a reversible looped transformer (~1M core) with per-iteration subspace reprojection, online-distilled from a ~135M teacher.
Author-email: Marius-Constantin Dinu <dinu.marius.constantin@gmail.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://dentate.cortex.a2olabs.com
Project-URL: Source, https://github.com/Xpitfire/spiral
Project-URL: Dentate, https://github.com/Xpitfire/dentate
Keywords: looped-transformer,reversible,distillation,small-language-models,dentate
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.6
Provides-Extra: train
Requires-Dist: transformers; extra == "train"
Requires-Dist: datasets; extra == "train"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Dynamic: license-file

# Spiral — a reversible looped transformer (~1M core)

> **One small block, iterated. Each pass folded into a new subspace. Reversible in time.**
> Spiral tests a single hypothesis: **a ~1M-parameter block iterated N times can recover the *reasoning depth*
> of a ~125M model** — online-distilled from a teacher (`SmolLM-135M`) that is also its baseline. Developed
> inside [Cortex](https://github.com/Xpitfire)/Dentate; usable standalone.

## The idea in one paragraph
A regular transformer stacks *D distinct* layers. Spiral has **one** shared block (`f_θ`, ~1M params) and runs
it for **N iterations**. To stop every iteration doing the *same* thing (provably weaker than untied depth —
Xu & Sato 2024), each pass is **folded into a different latent subspace** by an orthogonal rotation `Q_k` and
conditioned on the iteration index. The embedded input is **re-injected every step** (DEQ/Huginn).
The momentum coupling makes the augmented state algebraically invertible over the reals under fixed
deterministic replay. Floating-point reconstruction is approximate and configuration-dependent.
Current training runs a no-gradient prefix and an ordinary-autograd suffix (TBPTT), not reconstruction
backward; latent waists, modulation and external memory affect storage. Rotation, weight/gate normalization,
QK normalization and z-loss control selected scales but do not certify bounded states or a fixed point.
Zero transformer gates mean zero transformer force, not identity of the injected loop. Online knowledge
distillation and other objectives are selected by the trainer and recorded in the run configuration.

## Install
```bash
pip install -e .            # core (torch)
pip install -e '.[train]'   # + transformers/datasets for standalone distillation
pip install -e '.[dev]'     # + pytest/ruff
```

## Use
```python
from spiral import SpiralConfig, build_model
cfg = SpiralConfig(d_model=256, mean_iters=32, reproject="orthogonal", reversible="momentum",
                   teacher_model="smollm-135m")
model = build_model(cfg)            # lazy torch import; moves to cfg.device (mps)
logits = model(input_ids, n_iters=32)
model.num_params(core_only=True)    # 1,023,553
```
A config roundtrip preserves configuration, not a complete run: reproducibility additionally needs source/runtime, data/reference hashes, seeds, checkpoint/donor artifacts and decoding/masking metadata.

## Validate first, trust numbers later
`pytest` exercises configuration-specific contracts and numerical tolerance checks
(see [docs/VALIDATION.md](docs/VALIDATION.md)). Finite roundtrip, zero-transformer-force, output and gradient
checks are useful regressions, not universal precision, stability or nonvanishing-gradient guarantees.
```bash
pytest -q
```

## Historical scaffold observations (not independently reproduced for corrected code)
| Quantity | Value |
|---|---|
| Recurrent **core** params | **1,023,553 (1.024 M)** |
| Total (vocab 49,152, factorized E=128) | 7.315 M |
| Reported fp64 roundtrip | `max|Δh| = 1.6e-14`, `max|Δv| = 0`; incomplete replay/configuration provenance, not a universal bound |
| AdaLN-Zero force at init | `0.0` |
| Devices | CPU, MPS |

## Docs
- [docs/REQUIREMENTS.md](docs/REQUIREMENTS.md) — F/N spec: architecture, training loop, target, decisions.
- [docs/MATH.md](docs/MATH.md) — injected/folded/unfolded recurrence, real-arithmetic inverse, actual TBPTT
  versus proposed reconstruction backward, scale-control limitations and expressivity hypotheses.
- [docs/STABILITY.md](docs/STABILITY.md) — full augmented Jacobian, switched-product counterexamples,
  superseded historical diagnostic outputs and empirical controls without all-depth guarantees.
- [docs/SPIRAL_LATENT.md](docs/SPIRAL_LATENT.md) / [docs/SPIRAL_MEM.md](docs/SPIRAL_MEM.md) — the memory variants.
- [docs/SOURCES.md](docs/SOURCES.md) — verified references (arXiv IDs), grouped by design question (incl. §C.2
  gradient-dynamics and §E.2 memory-variant internals).
- [docs/VALIDATION.md](docs/VALIDATION.md) — the test/monitor-before-you-trust protocol (reconstruction,
  gradient health, theory-matches-practice).
- [docs/PLAN.md](docs/PLAN.md) — phased build (P0 scaffold ✅ → P6), risk register.

Canonical-paper headline metrics and figures are historical exploratory summaries, not independently
reproduced corrected-protocol results. Source repairs do not validate them retroactively; frozen artifacts,
causal/held-out recall controls and common evaluation references are required.

## Honesty
~1M coherence needs a **narrow distribution** (TinyStories 2023); broad-pool 100× knowledge compression is
unsupported (capacity gap). Spiral targets **narrow per-domain competence first** and treats broad-pool 100× as
the research hypothesis under test — not a promise. The bet is on *reasoning depth*, where looping, not
parameters, is the bottleneck. License: Apache-2.0.
