# Progression Problem v1 — full-lifecycle federated-twin demo trajectory

**Status:** Working roadmap. Reconstructed 2026-05-04 from Ben's verbal description after the stash-loss event lost the in-progress demo edits (~860 lines on `feat/coordinated-workloads`).

**Stub provenance:** the lost edits to `scripts/demo-three-approaches-scientist.py` (~648 lines) and `src/axiom/extensions/builtins/scidisplay/compute.py` (~212 lines) were *steps along this trajectory*, not the final v1. The base files on main are still useful starting points; the v1 builds on them.

---

## Thesis

A computational scientist with a model run can **decompose the run automatically across a federated tier of nodes** — local-cheap-fast → Rascal-medium-fast → TACC-when-required — and the federation will produce a result that is:

- **Faster** than running on any single tier alone (federation efficiency curve dominates the single-node curve as problem size grows),
- **More verifiable** than a black-box submission (every chunk carries signed provenance; the recomposer's aggregation is auditable),
- **More observable** than a queue submission (live instrumentation surfaces in a unified twin view, both during the run and as a post-facto record).

The progression problem demo is the *empirical substrate* for the "Above the Job Scheduler" paper. Without this trajectory, the paper has no benchmark; with it, the paper writes itself.

## v1 progression — start simple, layer complexity

The demo walks the audience up a four-rung ladder, each rung adding one capability the previous lacked:

### Rung 1 — single-node local

The simplest case. A small input model, single-process execution on the scientist's laptop. Zero federation, zero AI instrumentation. **Purpose:** establish a wall-clock baseline + sanity-check the input/output contract.

### Rung 2 — federated decomposition across local peers

Same input model, but the run decomposes across 2–4 federated peers on the local tier (other workstations, lab machines). Compute decomposition primitive (ADR-040) routes chunks; recomposer aggregates. **Purpose:** show the federation efficiency curve for trivially-parallel decomposition; demonstrate signed-provenance composition through the existing CompositionService.

### Rung 3 — federated decomposition crossing tiers (local + Rascal)

Bigger input model. Local peers handle the cheap leaves; Rascal handles the medium-cost leaves; trait-routing (ADR-040 D2) sends each chunk to the right tier. **Purpose:** show the trait-aware routing matching chunks to peer capability; demonstrate that crossing a trust/network boundary doesn't break composition.

### Rung 4 — full progression: local + Rascal + TACC, with twin instrumentation

Largest model — one that genuinely benefits from TACC capacity. Decomposition spans all three tiers. Live instrumentation reports per-chunk progress + emergent observables to a **whole-picture twin view** (the new twin feature). The view supports both:

- **Live queries** during the run: "which chunks are slowest? which peer is the long pole? what's the projected total wall-clock?"
- **Post-facto queries**: "for chunk C13, what was its peer's load when it ran? what did the recomposer assert about its result? what's the provenance chain to the final aggregated artifact?"

**Purpose:** prove the full-lifecycle case end-to-end — input model → federated decomposition → tier-aware routing → instrumented execution → unified twin view → signed final artifact. The demo terminates with the final artifact composed back into the user's memory substrate as a citable record.

## Required pieces — what exists, what's missing

| Piece | Status | Where |
|---|---|---|
| ADR-040 ComputeDecomposition primitive | ✅ landed | `docs/adrs/adr-040-compute-decomposition.md` |
| `spec-compute-decomposition.md` | ✅ landed | `docs/specs/` |
| Phase A scaffold of `compute_decomposition/` | 🟡 PR #142 (draft, post-recovery) | `feat/compute-decomposition-phase-a` |
| `cohort_model.py` (transient compute cohort) | ❌ not yet built | future task — see `2026-05-04-recovery-followups.md` |
| Coordinated workloads end-to-end primitive | 🟡 PR #141 (CI-green, queued) | `feat/coordinated-workloads` |
| Trust-weighted retrieval (signed provenance) | ✅ landed | `src/axiom/memory/trust_retrieval.py` (PR #140) |
| Scientific Displays compute kernel + signed artifacts | ✅ landed (Phase A) | `src/axiom/extensions/builtins/scidisplay/` |
| **Modern data platform extension** | ❌ designed but never built | new — required for v1 |
| **Twin extension (whole-picture live + post-facto view)** | 🟡 NeutronOS `feat/twin-extension-phase-1` | `Neutron_OS/` |
| **TRIGA digital twin data pipeline port** | ❌ not yet ported | NeutronOS — required for the live-twin-plus-actual-running demo |
| Three-approaches scientist demo (base) | ✅ on `feat/coordinated-workloads` | `scripts/demo-three-approaches-scientist.py` |

## Critical missing piece — the data platform extension

The unified twin view depends on a substrate that can ingest live instrumentation, persist signed observations, and serve both live and post-facto queries. Per Ben (2026-05-04): **"this should run on a new modern data platform extension that we've had designed for awhile but never built."**

Until that lands, the twin view either (a) runs on a temporary in-memory store (insufficient for post-facto queries beyond session lifetime), or (b) uses an existing component (e.g., the memory CompositionService for signed observations + a separate timeseries store for live signals — surfaces the seam).

**Action item:** locate the data-platform-extension design doc (likely in `docs/working/` or `docs/specs/`); confirm whether it's blocked on a decision, on implementation effort, or on dependency. If designed-but-deferred, escalate per the runway constraints.

## Critical missing piece — the twin extension's whole-picture view

NeutronOS has `feat/twin-extension-phase-1` in flight. The progression-problem demo needs the twin to support:

1. **Live mode** — subscribes to instrumentation streams from each running peer; aggregates into a unified view; supports queries against live state.
2. **Post-facto mode** — replays from signed observation history; supports queries against any past run.
3. **Live-plus-actual-running** — the near-future goal: TRIGA's actual-running data pipeline produces live observations alongside the digital twin's federation-decomposed model run; the view shows both side-by-side, allowing real-time discrepancy detection between model and reality.

The third mode is the killer demo. Until it ships, the v1 demo is "twin alone running across federated peers" — which is still novel, but loses the "actual + twin in parallel" wow factor that justifies the digital-twin framing.

## Sequence to v1

A workable execution sequence, dependency-ordered:

1. **Land the queued small PRs first** (PR #141 coordinated-workloads, PR #142 compute-decomposition phase A). These are the orchestration substrate.
2. **Build cohort_model.py** (per recovery-followups). Without it, the Rung 2-3 federation routing has no transient-cohort surface.
3. **Locate + decide on the data platform extension** (find the design doc, decide build-vs-defer).
4. **Land NeutronOS `feat/twin-extension-phase-1`** (already in flight). Get the live-mode twin view working against any signed observation source.
5. **Reconstruct the lost demo edits — but as the *Rung 2 → Rung 3* progression**, not as the original style. Use the now-on-main coordinated-workloads + compute-decomposition Phase-A primitives.
6. **Port TRIGA pipeline** (NeutronOS work). Closes the live-plus-actual-running gap.
7. **Empirical run** for the paper: choose a problem size that exercises all four rungs; produce the federation-efficiency curve + the per-tier breakdown.
8. **Demo + paper § 6** ship together.

## Why this informs the "Above the Job Scheduler" paper

The paper's §6 (empirical study) was originally proposed with three abstract benchmark candidates (particle method, parameter sweep, Monte Carlo ensemble). The progression-problem v1 supersedes those: it's a *concrete domain* (digital twin physics) executed *across the same domain-agnostic substrate*. The paper should reference this trajectory as its empirical substrate, with the indirect-framing care:

- The benchmark is "a representative deep-science model with progression-problem characteristics," not "TRIGA reactor physics specifically." TRIGA is one consumer; the framing belongs to the substrate.
- The federation efficiency curve plots local-only / local+Rascal / local+Rascal+TACC against problem size. The "above the job scheduler" claim is empirical at the third-tier crossover.
- The live-twin-plus-actual-running demo is the *narrative* of the paper, not its core claim — the core claim is the substrate primitive's efficiency curve.

## Open questions

- Does the data-platform-extension design doc still reflect Ben's current intent? If yes, it's the spec for the build. If not, it needs a refresh pass before implementation.
- Does the demo target the early-June Prague go-live, or post-Prague? (Per `project_prague_runway_plan_2026_04_29`, scidisplay implementation is *not* on the locked Prague critical path.) If Prague, the v1 needs to land in ~5 weeks; if post-Prague, the timeline is more relaxed and we can wait for the data-platform extension to land properly.
- Is the live-plus-actual-running demo the "wow" that defines v1, or v2? V1 with twin-alone is shippable sooner; v2 with actual+twin is the narrative we want to tell.
