The funnel, and what stopping means#

MatMelt’s pipeline is a funnel in the literal sense: it widens before it narrows. Understanding that shape explains the material counts you see, the EXPANDED state that confuses everyone once, and why stop_after behaves the way it does.

The five-stage funnel: compose and structure fan one seed out into several compositions and then several candidate crystals; screen narrows them, sending rejects to a terminal FAILED state; the optional calculate stage and then extract carry survivors to DONE. A stop_after setting rests materials at a stage rather than retiring them. The five-stage funnel: compose and structure fan one seed out into several compositions and then several candidate crystals; screen narrows them, sending rejects to a terminal FAILED state; the optional calculate stage and then extract carry survivors to DONE. A stop_after setting rests materials at a stage rather than retiring them.

Widening, then narrowing#

The first two stages fan out. One element combination becomes several plausible compositions; each composition becomes many candidate crystals, one per symmetry group tried. A single seed of [Na, Cl] becomes 37 candidate structures in about ten seconds, and a three-element seed with several dimensionalities requested becomes far more.

The remaining stages are 1:1, and screen is where the population shrinks: a candidate either relaxes to something physically sensible or it is rejected.

This shape is why cost is reported before work starts. The blast radius of an element set is combinatorial, and it is much cheaper to read 3 combination(s) in a preflight than to discover the number by watching a laptop for an afternoon.

Fan-out parents, and why EXPANDED is a success#

When a stage fans out, the original material has no further work of its own. It does not disappear — lineage matters — so it retires into EXPANDED.

EXPANDED is terminal, and it is emphatically not a failure. A run that ends with EXPANDED: 2, DONE: 28, FAILED: 9 produced 28 finished materials from two parents. Counting the parents as failures, or as finished materials, would be wrong in different directions; they are a third thing, so they get a third state.

Children are adopted idempotently: a parent that is settled twice does not produce duplicate children, and the parent only retires once every child has been adopted. A partially adopted fan-out leaves the parent claimable so the operation is simply redone — the alternative, a half-expanded parent marked finished, would lose materials silently.

Because children record their parent, run --materials <parent-id> reaches an entire lineage without you listing 37 ids.

What stop_after actually does#

stop_after names the last stage a run executes. The subtle part is what happens to the materials it leaves behind — and the answer changed, deliberately, after the original behaviour proved wrong in practice.

A mid-funnel stop rests materials. They sit at that stage’s output state, live and claimable. Raising stop_after, removing it, or running matmelt run --only <stage> continues them in the same workspace, with no regeneration and no surgery.

The rejected alternative was to mark them finished. That is superficially reasonable — the run did complete — and it is a trap: a stop_after: structure run over a large element set retired hundreds of materials as DONE, at which point no later run could claim them and continuing the study meant editing the database by hand. A pause must not be indistinguishable from a completion.

So the rule is narrow and mechanical: materials retire as DONE only when the run executes through the configured last stage. Which brings the one genuine gotcha:

Important

“Configured last stage” means the last stage in your config, not the last stage of the funnel. If you configure only compose and structure, then stop_after: structure is not a pause at all — the run went through everything it was given, and those materials retire as DONE. To keep them claimable, configure the stages you intend to run later.

--only <stage> follows the same rule from the other direction: a mid-funnel single-stage run advances materials to that stage’s output state and finalizes nothing, because later stages still have work to do.

The optional stage#

calculate is optional, and skipping it is not a special case bolted on afterwards. Each stage’s output state is declared, and the orchestrator derives each stage’s input state from the funnel you actually configured. With no calculate: block, screen completes a material straight to EXTRACTED and nothing ever parks in CALCULATED.

That is why a config written before the stage existed behaves identically today, and why the funnel accepts an ordered subsequence — calculate may be omitted, but nothing else may, because a missing required stage means the next stage’s inputs would never exist. That refusal names what is missing:

missing [structure]