T5 — Starting mid-funnel#

Goal: advance materials that already exist in a workspace without re-running what came before, and understand exactly how a stage decides whether it can work on a material at all.

Prerequisites: a workspace with materials resting mid-funnel — for example the one from Your first run, which leaves 36 candidates at STRUCTURED.

Time: about 15 minutes.

The idea#

Every stage is autonomous and contract-bound. It declares the files it needs and the files it produces, and before doing any work it validates its own inputs. Nothing in the tool assumes you arrived through the front door — which is what makes it safe to point a stage at a workspace built earlier, by an earlier run, or by a run that stopped halfway.

The other half of the promise matters just as much: when a stage cannot work on a material, it says so precisely rather than producing garbage.

1. See where everything rests#

$ matmelt status

Status of run resumed-run:
  Materials by state
┏━━━━━━━━━━━━┳━━━━━━━┓
┃ State      ┃ Count ┃
┡━━━━━━━━━━━━╇━━━━━━━┩
│ DONE       │     1 │
│ EXPANDED   │     2 │
│ STRUCTURED │    36 │
└────────────┴───────┘

Those 36 are live, claimable materials — a mid-funnel stop_after rests materials rather than retiring them. List them by state to pick targets:

$ matmelt status --state STRUCTURED

2. Advance one stage on demand#

--only runs exactly one stage over whatever is waiting for it, ignoring the rest of the funnel:

$ matmelt run --only screen

--only and pipeline.stop_after both answer “how far does this run go”, so they are mutually exclusive. Comment out stop_after before using --only:

run: --only and pipeline.stop_after are mutually exclusive -- remove stop_after
from the config or drop --only.

Combine it with --materials to advance a chosen few:

$ matmelt run --only screen --materials Cl1Na1_3D-3 --yes

When a mid-funnel --only run drains, materials rest at that stage’s output state. They are not retired as DONE, because later stages still have work to do on them:

--only extract: stage drained; downstream stages were not run.

3. What happens when an input is missing#

Now the contract. Suppose a candidate’s structure file went missing — a half-copied workspace, a cleanup script, a full disk mid-write. Ask screen to run on it anyway:

$ matmelt run --materials Cl1Na1_3D-1 --yes
$ matmelt status

┃ Material    ┃ Stage  ┃ Attempts ┃ Reason                                     ┃
┡━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Cl1Na1_3D-1 │ screen │        3 │ stage 'screen' cannot run on item          │
│             │        │          │ 'Cl1Na1_3D-1': missing or empty required   │
│             │        │          │ input file(s) ['structure.cif'] in         │
│             │        │          │ materials-work/Cl1Na1_3D-1 (one candidate  │
│             │        │          │ crystal from the structure stage)          │

The refusal names the stage, the material, the exact file, the directory it looked in, and what that file is for. No job was submitted; no MACE model was loaded; nothing was written.

Note Attempts: 3. A missing input is treated as a fault, not a verdict — an unmounted disk or a half-finished copy might genuinely be fixed by the time the next pass comes round — so it spent the full retry_budget before settling as FAILED. Compare that with a screened-out structure, which sits at Attempts: 1 because re-running it would give the same answer.

That difference is the whole verdict-versus-fault doctrine, visible in one column.

4. What happens when an input is present but wrong#

Contracts are not just about files existing. extract serves only materials that were screened in; hand it one whose verdict says otherwise:

$ matmelt run --only extract --materials Cl1Na1_3D-3 --yes

│ Cl1Na1_3D-3 │ extract │        3 │ stage 'extract' cannot run on item        │
│             │         │          │ 'Cl1Na1_3D-3': screen_result.json says    │
│             │         │          │ category 'FAILED', but extract serves     │
│             │         │          │ only RELAXED (screened-in) materials      │

The file was there and was valid JSON. The stage read it, understood it, and refused on the meaning — which is exactly what you want when starting mid-funnel with data you assembled yourself.

5. Fix and resume#

Repair whatever was actually wrong, then bring the failed materials back with retry:

$ matmelt retry --materials Cl1Na1_3D-3 --yes

Reset 1 material(s) -- they will re-run at their failed stage on the next
'matmelt run'.

retry resets FAILED materials to the state their timeline says they should resume from — the output state of the last stage they completed — clearing the attempt count and the failure reason while preserving the completion history. Preview before committing with --dry-run, or filter by stage:

$ matmelt retry --stage screen --dry-run

┃ Stage  ┃ Count ┃ Distinct reasons ┃ Example reason                           ┃
┡━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ screen │     1 │                1 │ stage 'screen' cannot run on item        │
│        │       │                  │ 'Cl1Na1_3D-1': missing or empty required │
│        │       │                  │ in...                                    │
retry --dry-run: nothing changed.

Then run again, and the repaired material advances:

$ matmelt run --only extract --materials Cl1Na1_3D-3 --yes

  Materials by state
┏━━━━━━━━━━━━┳━━━━━━━┓
┃ State      ┃ Count ┃
┡━━━━━━━━━━━━╇━━━━━━━┩
│ DONE       │     2 │
│ EXPANDED   │     2 │
│ FAILED     │     2 │
│ STRUCTURED │    33 │
└────────────┴───────┘

A note on where materials come from#

Materials enter a workspace through compose, which seeds them from your element set, and through fan-out, where a stage produces children. There is no import command for arbitrary external structures in this version — “starting mid-funnel” means continuing a workspace that MatMelt built, from wherever it currently rests.

What you learned#

  • --only <stage> advances exactly one stage; it is mutually exclusive with stop_after, and a mid-funnel run rests materials instead of retiring them.

  • A stage validates its own inputs before any work happens, and refuses by naming the stage, the material, the file and the reason.

  • Refusals come in two flavours — a missing file is a fault that spends the retry budget, while a wrong verdict or a rejected structure is settled once.

  • retry returns failed materials to their resume state, with --dry-run and --stage to preview and filter.

Next: T6 adds the optional DFTB+ stage. For the recovery workflow on its own, see Diagnose a failed material and Stop a run and continue it later.