# Above the Job Scheduler — Federation-Routed Compute as a Substrate Primitive

**Status:** Working draft (2026-05-04, reconstructed after a stash-loss event).
**Placement decision: TBD.** This may belong as a section of the locked Axiom paper #2 ("Memory That Federates" — substrate paper) or as a standalone Axiom-Compute companion piece. Reconstructed from ADR-040 + spec-compute-decomposition + portfolio-strategy memory; original draft intent was not fully captured. Resolve placement before promoting from `working/` into `papers/`.

---

## Abstract (draft)

The traditional job scheduler — Slurm, PBS, LSF, the long tail of HPC workload managers — assumes a single shared queue, a single trust boundary, and a single pool of compute. None of these assumptions survive contact with cross-institutional research. A computational scientist who wants to use a colleague's GPU at another university either (a) gets a guest account on that institution's cluster, (b) ships data to a commercial cloud, or (c) gives up. The federation peers — colleagues' workstations, lab machines, classmates' laptops — sit visible-but-unusable, even when the work decomposes naturally into chunks that a peer could verifiably execute.

We propose **federation-routed compute decomposition** as a substrate primitive — one tier above the job scheduler, parallel to federated memory and federated inference. The primitive defines a fixed vocabulary (`Problem`, `DecompositionPlan`, `Chunk`, `ChunkResult`, `Decomposer`, `Recomposer`, `LeafAdapter`) and a small closed registry of named decomposition patterns. Domain extensions parameterize the primitive with their pattern-specific decomposers, recomposers, and adapters. The primitive composes the decomposed work over an existing federation directory, executes leaves in per-language attestation sandboxes, and aggregates results through deterministic recomposers that surface signed provenance for every step.

The construction generalizes the "LLM proposes, deterministic kernel verifies" pattern that has already been validated on a different substrate (math rendering with SymPy authorization). Where traditional schedulers ask "where do I run this?", the primitive asks "how does this decompose, who can run each leaf, and how do we verify the aggregation is sound?" The answer is signed end-to-end and survives the federation as a citable scientific artifact rather than a transient queue submission.

## Why this is interesting now

Three independent forces converge:

1. **Compute is the largest unsolved primitive in the federation stack.** Memory propagates (federation directory, signed records). Inference routes (federated inference protocol). State replicates (signed gossip + DNS-style records). But a multi-hour deterministic-or-stochastic compute problem still binds to one node.

2. **The "propose, verify" pattern from scientific displays generalizes.** Math: LLM proposes an equation, SymPy authorizes the result, provenance composes. Compute: LLM proposes a decomposition plan, the decomposer authorizes the chunks, per-leaf adapters execute, the recomposer authorizes the aggregation. The pattern shape transports without modification.

3. **The substrate's other federation primitives are now in place.** Federation directory plus background tasks plus per-language attestation plus LLM-tier routing — every primitive a federation-routed compute capability needs already exists in some form. The decision is whether to compose them into a first-class primitive or let every consumer reinvent the orchestration.

The third force is the timing pressure. Without a unified primitive, the first three consumer extensions to want federation-routed compute will each invent their own decomposition vocabulary, their own seed-coordination scheme, their own aggregation protocol — and the federation directory will accumulate three incompatible record-type families. The cost of *not* unifying compounds quickly.

## Contribution

We contribute:

1. A **vocabulary** — eight types (`Problem`, `DecompositionPlan`, `Chunk`, `ChunkResult`, `Decomposer`, `Recomposer`, `LeafAdapter`, `PaperTemplate`) and a closed registry of named decomposition patterns — that any deep-science domain can parameterize.
2. A **trait-routing protocol** that matches chunks to peers using federation-directory traits (CPU class, GPU presence, classification clearance, liveness) without exposing peers' identities outside their classification scope.
3. A **closed loop** — propose, decompose, route, attest, execute, recompose, sign, distribute — that produces a citable artifact at the end. The artifact composes through the existing memory substrate; provenance is preserved without hand-marshaling.
4. A **comparison framework** against the traditional job scheduler that names the assumptions a scheduler bakes in (single queue, single trust boundary, single compute pool) and shows where the primitive transcends each.

We contribute *no* new sandbox technology, *no* new cryptographic primitives, *no* new federation transport. The work is entirely in composition: take what exists, name the primitive, fix the vocabulary, ship the registry.

## Indirect competitive framing

Per the portfolio strategy: never name commercial competitors directly. Frame against:

- **"Traditional cluster job schedulers"** — Slurm and family. Argue: assume a single trust boundary; cannot route across institutional lines; surface a queue, not a citable artifact.
- **"Commercial single-tenant compute platforms"** — the cloud-batch products. Argue: trade trust-fragmentation for cost; data-sovereignty failures at scale; no federation-native record of work performed.
- **"Hand-rolled per-domain orchestrators"** — the cottage industry of one-off Python scripts that distribute work across a few peers. Argue: each one re-invents decomposition vocabulary; each accumulates compatibility debt; none compose with each other.

The where-we-win moment: **at the second consumer**. With one consumer, a hand-rolled orchestrator is cheaper than a primitive. With two, the primitive amortizes; with three, hand-rolling produces three incompatible directory record-type families that cannot share peers. The advantage scales with the number of distinct decomposition patterns the substrate hosts — the federation directory is a multiplicative effect.

## Empirical substrate — the progression-problem v1

The paper's empirical claim ties to the **progression-problem v1 demo trajectory** (see `docs/working/progression-problem-v1-trajectory.md`). The trajectory is a four-rung ladder of increasing problem size against increasing federation tiers:

1. **Rung 1** — single-node local. Baseline wall-clock; sanity-check input/output contract.
2. **Rung 2** — federated decomposition across 2–4 local peers. Trivially-parallel decomposition, signed-provenance composition.
3. **Rung 3** — cross-tier (local + Rascal). Trait-aware routing matches chunks to peer capability; trust/network-boundary crossing.
4. **Rung 4** — full progression (local + Rascal + TACC) with live twin instrumentation. Whole-picture observation supporting both live and post-facto queries; signed final artifact composes back into the memory substrate.

The federation-efficiency curve plots wall-clock vs. problem size for each rung. The "above the job scheduler" claim is empirical at the Rung-3 → Rung-4 crossover: a single-trust-boundary scheduler can't cross institutional lines, so the curve flatlines or fails entirely; the substrate's curve continues to dominate.

**Indirect framing per portfolio policy:** the benchmark is "a representative deep-science model with progression-problem characteristics," not any specific domain consumer. The trajectory's narrative-grade demo (live twin + actual running, e.g., a digital-twin pipeline alongside its model run) is the *positioning* — the core claim is the substrate primitive's efficiency curve, which is domain-agnostic.

Backup candidate benchmarks if the progression-problem trajectory slips for runway reasons:

- **Parameter-sweep decomposition over a published BEIR or SciFact retrieval-evaluation suite** — pure decomposition-and-aggregation; no domain coupling.
- **Ensemble-run decomposition over a published Monte Carlo problem** — least domain-specific; cleanest for a pure-substrate venue.

## Sections to flesh out next session

- §3 Decomposition pattern registry — named patterns + invariants the recomposer restores
- §4 Trait routing — the matchmaking algorithm; classification-aware peer selection
- §5 The closed loop — sequence diagram of propose→decompose→route→attest→execute→recompose→sign→distribute
- §6 Empirical study — choose benchmark, run, plot the federation-efficiency curve
- §7 Comparison to job schedulers — assumption-by-assumption breakdown
- §8 Threat model — what changes vs. single-trust-boundary scheduling

## Notes for whoever picks this up

- The body of the technical claims lives already in `docs/adrs/adr-040-compute-decomposition.md` and `docs/specs/spec-compute-decomposition.md`. This paper is the *positioning* of those claims, not new substance.
- The `cohort_model.py` work that was lost is the runtime enabler for §3-§4 (the cohort is the routed-to peer set per chunk). If/when that lands, this paper acquires its empirical substrate.
- Domain-agnostic per `feedback_axiom_domain_agnostic`. Any reference to "reactor" / "facility" / "course" content moves to a domain-extension companion piece.
