A trainable, instrumented architecture that replaces the transformer's quadratic attention with a cellular-sheaf diffusion: the data's geometry is recomputed and jammed in every forward pass, a multi-scale band bank reads the transient, and a sparse route head restores sharp retrieval.
Pure NumPy implementation (no torch, no scipy) so that every claim can be re-verified on any laptop in under a minute. Numbers on this page are generated from the measurement files — nothing is hand-typed.
This is the "take it to the max" table. Each row is a named failure mode from the original proposal, the module that fixes it, and the mechanism — with the measured effect wherever the effect is measurable.
Five lemmas carry the architecture. selftest.py verifies each one numerically and writes the results to results/selftest.json.
Project Gutenberg (Alice in Wonderland), char-level, identical tokens, identical optimiser, identical schedule.
The label is the sequence mean — exactly the quantity the Hodge limit computes. A local MLP with a fixed receptive field cannot see it; the fluid and attention both can.
Everything above is theory. This is the experiment that settles the central claim. The input is a sequence of scalars encoded as fixed features; the target is the mean — a global, order-invariant statistic that a strictly local model cannot compute. Only the aggregation mechanism differs between the arms; nothing else is trainable.
Chance = 50%. The fluid reaches 97.3% in one pass — it beats both strictly-local aggregators and nearly matches attention. This is the mechanism claim, measured.
Every one of these was found by a test, not by reading the code. They are listed because they are the honest cost of a from-scratch implementation — and because each one had silently turned a core claim into a no-op.
| bug | symptom | fix |
|---|---|---|
Laplacian sign (A−D instead of D−A) | non-PSD operator, overflow in exp(−tL) | correct construction |
| Chebyshev recurrence missing the factor 2 | expansion plateaued at 78% error, independent of degree | Tk = 2L̃Tk−1 − Tk−2 |
index_add never scattered in the forward pass | the whole fluid was inert — the block's Laplacian was identically zero | scatter in the forward pass + a regression test |
index_add backward never reached src | restriction maps received no gradient at all | gather the incoming gradient at the scattered rows |
gradients zeroed after backward() | both models frozen at ln(vocab) for 400 steps | zero before the forward pass |
exp(+α) instead of exp(−α) in the coefficients | bands multiplied by e2α ≈ 10⁶ → divergence | sign fix + a stability regression test |
mean() overwrote the sum's backward | gradients silently never reached the inputs | chain instead of replace |
no gradient seeding in backward() | every gradient exactly zero | seed the output gradient |
newaxis ([:, :, None]) unhandled in the backward | scatter with mismatched index arrays | strip and squeeze newaxis axes |
| λmax underestimated by power iteration | Chebyshev recurrence outside its stable region | Gershgorin upper bound (differentiable) |
Every block reports its spectral gap, G-balance residual, band energies and persistent signature on every forward pass. This is the framework's real advantage over a black-box: the failure mode is visible before the loss curve moves.
Energy carried by each band of the filter bank. A collapse onto one band means the multi-scale readout has degenerated into a single exponential.
λmax is the spectral scale (sets the band placement); balance residual ‖L1‖∞ is the G-balance indicator — the phase transition; α_neg_frac is how "signed" the learned geometry is.
The fluid is 0.00016 % of a transformer layer. It is nearly free — which also means it cannot be the reason the model works.
| Context | Transformer KV cache | FluidLayer (static topology) |
|---|---|---|
| 4,096 | 2.0 GB | 16 KB |
| 32,768 | 16.0 GB | 16 KB |
| 131,072 | 64.0 GB | 16 KB |
| 1,000,000 | 488 GB | 16 KB |
Param-matched full training run (D ≈ 44B, 470B tokens, 512×H100 @ 40 % MFU): 1.10× cheaper at 8K context, 2.54× at 128K.
15/15 numerical checks; end-to-end autodiff with gradient checks; LM A/B; mechanism demo.
The dense O(n²) Laplacian is the current scaling limit. A1–A12 ablations produce one results table.
Place the bands around the measured 1/λ₂ — the critical temperature becomes a hyperparameter.
Syntax / semantics / topology as separate sheaves with learned mixing; differentiable complex construction; molecules & proteins, where the prior is not a gimmick.
FluidLayer Framework v1.0 — generated by build_report.py from results/selftest.json and results/train_results.json. Full specification in FRAMEWORK.md; theory & cost analysis in the companion report FluidLayer v0.1.