# Pipeline Authoring Model

**Status:** DESIGN — all open questions resolved, implementation not started
**Date:** 2026-03-03
**Roadmap:** T3-A1 through T3-A7 (`ROADMAP.md`)

## Related Documents

- [Connector Interface](connector-interface.md) ← how connectors plug into the harness
- [Python API](python-api.md) ← `@pipeline`, `@phase`, `stratum.run()`, phase result schema
- [Compose + Stratum + spec-kit Substrate](../compose-speckit-substrate/design.md) ← predecessor design, T3-1 through T3-9
- [`.compose.yaml` Design](../../../coder-compose/docs/plans/2026-03-03-compose-yaml-design.md) ← earlier sketch, superseded by this doc
- [Stratum Roadmap](../../../ROADMAP.md) ← Track 3

---

## Problem

The natural question when designing a pipeline-as-code system is: what file format do users
write? We spent time designing `.compose.yaml` before asking a more important question: do
users write a pipeline spec at all?

They don't. And they shouldn't have to.

`.stratum.yaml` is IR. It is generated by the library and by Claude. Nobody writes it from
scratch — this is already established in the project. The same logic applies to any
high-level pipeline spec: if you need to author a structured YAML file to describe phases
and policies, that is a tool problem, not a user workflow.

There are three kinds of Stratum users. None of them need to write a pipeline spec file.
A fourth interface — `stratum-ui` — provides a visual editor that generates the markup
for any of the three formats above, and doubles as the gate approval surface.

---

## Three User Interfaces

### 1. Python Developers — `@pipeline` / `@phase` decorators

Developers using `stratum-py` already write Python. The pipeline is expressed as Python:

```python
from stratum import pipeline, phase, Capability, Policy

@pipeline(name="feature-lifecycle")
class FeaturePipeline:

    @phase(capability=Capability.SCOUT, policy=Policy.SKIP)
    async def discovery(self, feature: str) -> DiscoveryResult:
        """Explore codebase, identify affected files and dependencies"""
        ...

    @phase(capability=Capability.CRITIC, policy=Policy.GATE, input=["discovery"])
    async def pre_gate(self, discovery: DiscoveryResult) -> GateResult:
        """Assess readiness — conflicts, blockers, unclear requirements"""
        ...

    @phase(capability=Capability.BUILDER, policy=Policy.SKIP, input=["discovery", "pre_gate"])
    async def implement(self, discovery: DiscoveryResult, gate: GateResult) -> ImplResult:
        """Implement the feature"""
        ...

    @phase(capability=Capability.CRITIC, policy=Policy.FLAG, input=["implement"])
    async def post_gate(self, impl: ImplResult) -> ReviewResult:
        """Review implementation quality"""
        ...

    @phase(capability=Capability.BUILDER, policy=Policy.GATE, input=["implement", "post_gate"])
    async def checkpoint(self, impl: ImplResult, review: ReviewResult) -> None:
        """Confirm complete, capture learnings"""
        ...
```

The library compiles this to the execution IR at decoration time. The user never sees
`.stratum.yaml`. The Python class is the spec.

### 2. Claude Code Users — skill invocation

Users running Claude Code invoke a skill. The pipeline is defined once inside the skill;
the user just provides the feature description:

```
/stratum-feature "add dark mode toggle to settings panel"
```

No file. No spec. The skill embodies the pipeline. The pipeline runs.

### 3. Teams / Operators — `stratum.toml`

Teams that want to override policy defaults for their project write a small config file.
This is not a pipeline spec — it is a policy overlay. The pipeline is still defined in code
or skill. The config controls behavior:

```toml
[pipeline.policy]
pre_gate = "flag"      # default is gate; this team prefers non-blocking
post_gate = "gate"     # always gate review

[pipeline.capabilities]
scout  = "haiku"       # model hint, connector resolves
builder = "sonnet"
critic  = "sonnet"

[pipeline.connector]
default = "claude-code"
implement = "codex"    # use Codex for implementation phases
```

`stratum.toml` is optional. Sensible defaults exist without it.

---

## Pipeline Model

The concepts shared across all three interfaces.

### Phases

A named unit of work with:
- **intent** — plain-language description of what must happen
- **capability** — what kind of agent is needed (abstract tier, not a model)
- **policy** — gate / flag / skip (Compose's 3-mode dial)
- **input** — which prior phase outputs this phase consumes
- **ensures** — postconditions that must hold before the phase is considered complete

### Capability Tiers

Abstract roles. Connectors resolve them to concrete agents and models.

| Tier | Role |
|---|---|
| `SCOUT` | Fast, read-only exploration — identify files, context, risks |
| `BUILDER` | Full capability — writes code, edits files, runs tests |
| `CRITIC` | Review and assessment — reads output, evaluates quality |

Three tiers is enough for the lifecycle. Tiers are extensible if a meaningful distinction
emerges (e.g., `PLANNER`) but the bar for adding a tier is high — it must require a
meaningfully different agent profile.

### Policy Modes

| Mode | Behavior |
|---|---|
| `GATE` | Block until human explicitly approves — phase does not advance |
| `FLAG` | Proceed, notify human, log decision for audit |
| `SKIP` | Proceed silently, record for audit trail |

Policy is the gate/flag/skip dial from Compose's core axiom. It is inherited downward
through the pipeline hierarchy and overridable at any level.

### Agent Interleaving

Each phase can route to a different agent. The connector resolves `capability` to a
concrete agent/model. A per-phase connector override allows explicit routing:

```python
@phase(capability=Capability.BUILDER, policy=Policy.SKIP, connector="codex")
async def implement(self, ...):
    ...
```

Or in `stratum.toml`:
```toml
[pipeline.connector]
implement = "codex"
```

This is sequential interleaving — not parallel coordination. Each phase delegates to one
agent and waits for its result. The capability tier is the safe abstraction: `BUILDER`
doesn't mean "use Codex," it means "I need a builder." The connector decides what that
means. You can swap connector mappings without touching the pipeline definition.

### Output Passing

Phase outputs are written to disk in a deterministic run workspace:

```
.stratum/runs/{run-id}/{phase-id}.json
```

- `run-id` is generated when the pipeline starts for a given feature
- `{phase-id}.json` contains the structured output matching the phase's output contract
- The connector writes the file after phase completion — the agent does not need to know
  the workspace convention
- Before starting phase N, the connector reads all `input` files and injects their
  contents as context for the next agent

This survives session death. If a session dies mid-pipeline, completed phase outputs
remain on disk. The next session resumes from the first incomplete phase.

---

## Execution Model

### The State Machine Primitive

The fundamental unit of Stratum execution is a single state machine transition:

```
read state → determine next phase → execute → write result → evaluate ensures → advance
```

This is the only thing that needs to exist. Everything else is a driver for this loop.

State lives in `.stratum/runs/{run-id}/` — output files, gate files, verification results.
The state machine reads from and writes to this directory. It has no other dependencies.

### Drivers

The same state machine is driven in different ways depending on context:

| Driver | How it works |
|---|---|
| **Python harness** | `stratum.run(Pipeline, ...)` — loops internally until complete |
| **MCP step-by-step** | Claude Code calls `stratum_plan` / `stratum_step_done` — drives one step per tool call |
| **CLI invocation** | `stratum run --next` — reads state, executes one step, exits; caller invokes again |
| **File-watch loop** | Process watches `.stratum/runs/`, invokes step on state change |

These are not different systems. They are different invocation models for the same
state machine. The state machine is indifferent to who is driving it.

### Unification of stratum-py and stratum-mcp

`stratum_plan` / `stratum_step_done` is Claude Code driving the state machine one step
at a time via MCP tool calls. `stratum.run()` is the Python harness driving the same
state machine as an internal loop. Same transitions, same state files, same ensure
evaluation — different drivers.

The MCP interface is not a separate system. It is an API surface onto the state machine
that lets Claude Code be the driver. The Python harness is the same state machine with
the driver built in.

### No Daemon Required

A daemon is just the file-watch driver — the state machine invoked on file change events.
It is not a distinct architectural component. If you need background execution, run the
Python harness as a process. If you need step-by-step control, use the MCP interface.
The daemon adds no capability that isn't already present in the harness.

---

## What `.stratum.yaml` Means Now

Unchanged. `.stratum.yaml` is the execution IR generated by the library and by Claude.
Nobody writes it. The pipeline authoring interfaces (Python decorators, skills, `stratum.toml`)
compile down to it.

The earlier debate about naming the high-level pipeline spec is resolved by recognizing
there is no user-facing spec format. The IR keeps its name. Users interact with Python,
skills, and `stratum.toml`.

---

## `stratum-ui`

`stratum-ui` is a separate project — a first-party reference UI for Stratum. It is a
human-facing application with its own HTTP server. It is not an MCP client. MCP is the
protocol between Claude Code and Stratum tools; the viewer is the protocol between humans
and Stratum state.

```
Claude Code  →  stratum-mcp (MCP tools)  →  .stratum/runs/, stratum.toml
Human        →  stratum-ui (HTTP UI)  →  .stratum/runs/, stratum.toml
```

Both clients share the same data layer — files on disk. Neither needs to know about the
other. The file-based conventions are the coordination mechanism.

### Four responsibilities

**Monitor** — pipeline runs, step state, phase completion, audit traces from
`stratum_audit`. The read view of `.stratum/runs/`.

**Gate** — the primary gate approval surface. Watches `.stratum/runs/{id}/{phase}.gate`
files. Surfaces pending gates in a queue with phase context and agent output. Writes
`.gate.approved` (or `.gate.rejected`) when the human acts. This resolves OQ-2: the
gate approval channel is file-based; the viewer is the canonical UI on top of it.

**Edit** — visual pipeline definition. Configure phases, capability tiers, policy dials,
per-phase connector overrides, phase ordering. No YAML or Python required.

**Generate** — export the visual definition to:
- `stratum.toml` — policy/capability overrides for an existing pipeline
- Python `@pipeline` class — full class definition, drop into a project
- `.stratum.yaml` IR — for direct execution without a source file

### What the viewer does not do

It does not talk to stratum-mcp. stratum-mcp is Claude's interface. The viewer talks
directly to the filesystem (or to a thin local HTTP server that mediates filesystem access
if running in a browser context). No MCP, no AI protocol, no tool calls.

---

## Open Questions

### OQ-1: Output Passing — RESOLVED

Phase outputs are written to `.stratum/runs/{run-id}/{phase-id}.json` by the connector
after phase completion. The connector reads `input` files and injects their contents as
context before spawning the next agent. Agents do not need to know the workspace convention.
Session death is safe — completed phase outputs persist on disk and the next session resumes
from the first incomplete phase. See Output Passing section above.

### OQ-2: Gate Approval Channel — RESOLVED

`policy=GATE` uses a file-based protocol. The connector writes a `.gate` file when blocked;
the viewer (or any other watcher) writes `.gate.approved` or `.gate.rejected` to unblock.

```
.stratum/runs/{run-id}/{phase-id}.gate          ← connector writes on block
.stratum/runs/{run-id}/{phase-id}.gate.approved ← viewer writes on human approval
.stratum/runs/{run-id}/{phase-id}.gate.rejected ← viewer writes on human rejection
```

The connector polls for the response file. No runtime coordination between connector and
viewer required. Works without stratum-ui (CLI tools, scripts, or manual file writes
can approve). stratum-ui provides the canonical UI on top of this protocol.

### OQ-3: Ensures Portability — RESOLVED

Two-tier ensures. Named assertions are portable across all connectors. Arbitrary Python
expressions are available but Claude Code only.

**Tier 1 — Named assertions (portable, default)**

A fixed vocabulary any connector can evaluate natively:

| Assertion | Meaning |
|---|---|
| `tests_pass` | `result.tests_pass == True` |
| `lint_clean` | `result.lint_clean == True` |
| `files_changed` | `len(result.changed_files) > 0` |
| `approved` | `result.approved == True` |
| `no_issues` | `len(result.issues) == 0` |
| `file_exists(path)` | file exists on disk |
| `file_contains(path, s)` | file contains substring |

These map 1:1 to existing stratum-mcp builtins when compiled to `.stratum.yaml`.
For other connectors, evaluated natively. No new infrastructure required.

```python
@phase(capability=Capability.BUILDER, policy=Policy.SKIP,
       ensures=["tests_pass", "files_changed"])
async def implement(self, ...):
    ...
```

**Tier 2 — Arbitrary expressions (Claude Code only)**

Pass-through strings compile directly to `.stratum.yaml` ensures. Work only when
stratum-mcp is the evaluator. The `@pipeline` compiler emits a warning at decoration
time if arbitrary expressions are used alongside a non-Claude connector config.

```python
@phase(capability=Capability.BUILDER,
       ensures=["result.coverage > 0.8"])  # Claude Code only
```

**Why not a shared evaluator endpoint?** It introduces a runtime dependency —
every connector requires stratum-mcp to be running and reachable. Named assertions
avoid this entirely. The vocabulary is the contract; connectors implement it locally.

**Vocabulary stability:** The named assertion set is a stability surface. Additive only.
New assertions require every connector to implement them — bar is high.

---

## Roadmap Items

| ID | Item | Status |
|---|---|---|
| T3-A1 | Pipeline authoring model design (this doc) | COMPLETE |
| T3-A2 | `@pipeline` / `@phase` Python decorators in `stratum-py` | PLANNED |
| T3-A3 | Capability tiers (`Capability`, `Policy` enums) in `stratum-py` | PLANNED |
| T3-A4 | `stratum.toml` project config — policy overrides, capability mapping | PLANNED |
| T3-A5 | Run workspace convention — `.stratum/runs/` output passing | PLANNED |
| T3-A6 | Skill-driven pipeline execution — no YAML visible to user | PLANNED |
| T3-A7 | File-based gate protocol — `.gate` / `.gate.approved` convention | PLANNED |
| T3-A8 | `stratum-ui` — monitor, gate, edit, generate (separate project) | PLANNED |
