---
description: The separation the whole architecture hangs from — a stage decides what to compute, a backend decides where, and neither knows the other's business.
---

# What vs. where

The rule everything else follows: **a stage decides *what* to compute; a backend
decides *where* it runs.** Neither knows the other's business, and the only thing
that crosses between them is a portable description of a job.

```{image} /_static/diagrams/what-vs-where-light.svg
:alt: A stage builds a JobSpec that names no machine; either backend can run it, and the result comes back for the stage to judge. An arrow from the stage to a host is marked as never permitted.
:class: only-light
:width: 100%
```

```{image} /_static/diagrams/what-vs-where-dark.svg
:alt: A stage builds a JobSpec that names no machine; either backend can run it, and the result comes back for the stage to judge. An arrow from the stage to a host is marked as never permitted.
:class: only-dark
:width: 100%
```

## The thing in the middle

A stage's whole output is a `JobSpec`: an argv to execute, the files the job
needs, the files it promises to produce, an environment, a working directory,
and a coarse statement of how much machine it wants. That is all.

What a `JobSpec` deliberately does **not** contain is a hostname, a username, an
SSH key, a queue name, a scheduler directive, or an absolute path on somebody
else's machine. It cannot express those things, which is what makes it portable:
the same spec is runnable by a backend that forks a subprocess and by one that
rsyncs to a cluster and calls `sbatch`.

The red arrow on the diagram is the point. No stage anywhere in the codebase
reaches a machine. If one did, moving that stage to a cluster would become a
code change, and the property this whole design exists to protect would be gone.

## Why it is worth the indirection

Because the interesting question in a materials funnel is *where the expensive
stage runs*, and that answer changes constantly — laptop today, the group's GPU
box next week, a SLURM allocation for the big run. With the split, that is one
line of YAML:

```yaml
stages:
  screen:
    backend: cluster
```

Without it, it is an edit to the screening code, a new code path to test, and a
second way for the stage to be wrong. The stage does not become more correct by
knowing about SSH; it becomes harder to test and impossible to reuse.

The same split is what lets the test suite drive the entire pipeline with a fake
backend that never runs anything, and what lets every stage's real work be run
by hand as a plain script when you are debugging.

## The other half: backends never judge science

The split cuts both ways, and the second half is the one people get wrong.

A backend runs a job and reports what happened. It does **not** decide whether
what happened was good. A job that exits non-zero has not failed as far as the
backend is concerned — it ran, and its exit code was 3. That comes back as an
ordinary `JobResult` for the stage to interpret.

Backends raise only for **infrastructure** problems: the host is unreachable,
the scratch directory is not writable, the spec was malformed. The distinction
is not pedantry — it decides what the tool does next. A scientific rejection
should retire a material permanently; an unreachable host should pause the stage
and try again later. Conflating them either retries physics forever or
permanently discards a material because a network blipped.

There is a third state worth naming, because it is a real source of silent bugs
elsewhere: a job that ran but whose exit code could not be recovered reports
`None`, never `0`. "I don't know" and "it succeeded" are different answers, and
defaulting the first to the second is how a broken job becomes a fabricated
result.

## What this buys you, concretely

- **Backend choice is configuration.** The registry maps a `kind` string to an
  implementation; adding a backend adds a config option, not a branch in a stage.
- **Stages are testable without compute.** A fake backend fabricates the declared
  outputs, so a stage's logic is exercised with no science stack installed.
- **A backend is testable without science.** Its contract is "run this argv,
  bring back these files", which is checkable with `/bin/echo`.
- **The same conformance tests run against every backend**, so a new one either
  satisfies the contract or fails a test that already exists.

## Related

- {doc}`stage-contract` — what a stage owes in exchange for this freedom.
- {doc}`remote-execution` — what the `ssh_exec` backend does with a `JobSpec`.
- {doc}`../reference/config` — where `backends:` and `stages:` are written down.
