Built-in stages#

The five funnel stages. Each is a thin class whose science lives in a self-contained runner script.

The compose stage: one element-combination seed -> N candidate compositions (3.1).

The thin Stage half of the pair; the science lives entirely in compose_runner.py (self-contained: stdlib + smact, zero matmelt imports), which this class ships to the backend as an input artifact of every job. This module itself imports no science – it builds job descriptions and judges results, nothing more, so it loads on any machine.

The seed needs no input file: its id is the data. Seed ids come from core/seeds.py ("Cl-Na", "Cl-Na-O" – sorted element symbols joined by -), and build_jobspec decodes the id back into --elements arguments. A malformed id is refused clearly by validate_inputs.

Fan-out contract: the runner writes children.json (declared in produces) plus one children/<formula>/composition.json per charge-balanced composition, all in the job’s cwd; the orchestrator adopts the children and retires the seed as EXPANDED. Zero children is a valid result – some combinations simply have no charge-balanced compound.

matmelt.core.stages.compose.RUNNER_NAME = 'compose_runner.py'#: The self-contained script shipped with every compose job. Lives next to this module inside the installed package; staged into the job’s cwd by the backend, so the command references it by bare name.

class matmelt.core.stages.compose.ComposeStage(python='python3', threshold=2, max_unary_atoms=None)[source]#

Bases: Stage

Turn one element-combination seed into candidate-composition children.

Parameters:

python (str, default: 'python3') – Interpreter to invoke on the backend this stage is wired to (the user’s own environment holds the science stack – never bundled). Plain python3 by default; a config option, never a hardcoded compute detail.
threshold (int, default: 2) – Maximum stoichiometric coefficient SMACT explores. Higher finds more exotic stoichiometries and costs combinatorially more.
max_unary_atoms (int | None, default: None) – Cap for single-element seeds: the runner emits the elemental compositions X1..Xn (allotrope stoichiometries – SMACT’s charge balance cannot apply to one element). None (the default) follows threshold; set it separately to reach e.g. C60 without exploding SMACT’s multi-element search.

Raises:

StageError – For any invalid option. Every value is checked at construction, so a config mistake surfaces when the run is wired rather than as N failed jobs later.

fan_out: bool = True#: Whether this stage expands its item into children (see class docstring) instead of advancing it. Class attribute – override per subclass.

runtime_requirements: tuple[Requirement, ...] = (Requirement(module='smact', pip_name='smact', purpose='charge-balance filtering of element combinations (compose runner)'),)#: The science packages this stage’s runner imports, checked by the preflight before any work starts. Class attribute – a science stage overrides it; the default (nothing beyond stdlib) is itself a contract, e.g. extract’s.

property name: str#

Stable stage identifier – also the per-item idempotency key in state.

Returns:: The canonical name ("compose", "screen", …). It is what a config’s stages: block names, what the stage registry maps, and what the state store’s completion ledger records per item, so completing a stage twice is a no-op.
Return type:: str

property requires: StageInputSpec#

What must be true of an item’s directory before this stage runs.

Returns:: The declared input contract, enforced by validate_inputs.
Return type:: StageInputSpec

property produces: StageOutputSpec#

What this stage guarantees an item’s directory contains on success.

Returns:: The declared output contract, enforced by validate_outputs.
Return type:: StageOutputSpec

validate_inputs(item)[source]#

Refuse clearly when the seed id does not decode into element symbols.

Extends the base file check, which is vacuous here: this stage declares no input files, because the seed id is the data.

Parameters:: item (WorkItem) – The seed about to be composed.
Raises:: StageInputError – When the id is not element symbols joined by -.
Return type:: None

static elements_of(item)[source]#

Decode a seed id into its element symbols.

Parameters:: item (WorkItem) – The seed, whose id looks like "Cl-Na-O".
Returns:: The symbols, order preserved as-is – seed ids from core/seeds.py are already sorted.
Return type:: list of str
Raises:: StageInputError – Naming exactly what is wrong: an empty or non-symbol part (symbols are [A-Z][a-z]{0,2}), or a duplicated element.

build_jobspec(item)[source]#

Build the portable job description for one item.

The only place in a stage where a command is constructed. The spec names no host, queue, or transport; paths point into item.dir; any stage options (model path, device, thresholds) were injected at stage construction from config – never hardcoded here.

Parameters:: item (WorkItem) – The item to build a job for. Its inputs have already passed validate_inputs.
Returns:: A portable job description any backend can execute.
Return type:: JobSpec
Raises:: StageError – If the stage is misconfigured such that no job can be built at all – a whole-stage fault, never a per-item verdict.

interpret_result(item, result)[source]#

Judge a finished compose job.

Parameters:

item (WorkItem) – The seed whose job finished.
result (JobResult) – What the backend reported.

Returns:

ok on exit 0, which includes a fan-out into zero children. Anything else is a fault carrying the runner’s stderr tail, and never terminal: a compose failure is an environment problem – missing smact, a broken interpreter – that a retry can genuinely resolve once the user fixes the venv. The runner reserves science judgments for exit 0.

Return type:

StageVerdict

The structure stage: one composition -> M candidate crystal structures (3.2).

The thin Stage half of the pair; the science lives entirely in structure_runner.py (self-contained: stdlib + pyxtal, zero matmelt imports), which this class ships to the backend as an input artifact of every job. This module itself imports no science – it builds job descriptions and judges results, nothing more, so it loads on any machine.

The input is the composition.json a compose child carries (formula, elements, stoichiometry). validate_inputs goes beyond file presence and checks the file actually decodes into that shape – someone starting the pipeline mid-way with a hand-made composition is refused clearly, before a job is built, never with silent garbage or a confusing runner crash.

Fan-out contract: the runner tries every group of each requested dimensionality (--dims; 3D=230 space, 2D=80 layer, 1D=75 rod groups) and writes children.json (declared in produces) plus one children/<Formula>_<DIM>-<SG>/ directory per valid candidate (holding structure.cif + structure.json), all in the job’s cwd; the orchestrator adopts the children and retires the composition as EXPANDED. Zero children is a valid result – some compositions satisfy no symmetry constraints at all.

matmelt.core.stages.structure.RUNNER_NAME = 'structure_runner.py'#: The self-contained script shipped with every structure job. Lives next to this module inside the installed package; staged into the job’s cwd by the backend, so the command references it by bare name.

matmelt.core.stages.structure.COMPOSITION_NAME = 'composition.json'#: The input file a compose child carries – declared as an input artifact so the Phase-4 staging transport knows to push it (it already sits in the item dir locally; backends stage in-place inputs as a no-op).

class matmelt.core.stages.structure.StructureStage(python='python3', dimensions=('3D',))[source]#

Bases: Stage

PyXtal-generate symmetry-constrained candidates for one composition.

Parameters:

python (str, default: 'python3') – Interpreter to invoke on the backend this stage is wired to (the user’s own environment holds the science stack – never bundled). Plain python3 by default; a config option, never a hardcoded compute detail.
dimensions (Any, default: ('3D',)) – Which dimensionalities the runner generates – any of 3D (230 space groups), 2D (80 layer groups), 1D (75 rod groups). A single tag string or a list; defaults to ("3D",) (the pre-existing behaviour). A config option, forwarded to the runner as --dims.

Raises:

StageError – For any invalid option, so a config mistake surfaces when the run is wired rather than as N failed jobs later.

fan_out: bool = True#: Whether this stage expands its item into children (see class docstring) instead of advancing it. Class attribute – override per subclass.

runtime_requirements: tuple[Requirement, ...] = (Requirement(module='pyxtal', pip_name='pyxtal', purpose='symmetry-constrained random crystal generation (structure runner)'),)#: The science packages this stage’s runner imports, checked by the preflight before any work starts. Class attribute – a science stage overrides it; the default (nothing beyond stdlib) is itself a contract, e.g. extract’s.

property name: str#

Stable stage identifier – also the per-item idempotency key in state.

Returns:: The canonical name ("compose", "screen", …). It is what a config’s stages: block names, what the stage registry maps, and what the state store’s completion ledger records per item, so completing a stage twice is a no-op.
Return type:: str

property requires: StageInputSpec#

What must be true of an item’s directory before this stage runs.

Returns:: The declared input contract, enforced by validate_inputs.
Return type:: StageInputSpec

property produces: StageOutputSpec#

What this stage guarantees an item’s directory contains on success.

Returns:: The declared output contract, enforced by validate_outputs.
Return type:: StageOutputSpec

validate_inputs(item)[source]#

Refuse clearly when composition.json does not decode into a composition.

Extends the base file-presence check with a schema check, so a mid-pipeline start on a hand-made composition is refused before any job.

Parameters:: item (WorkItem) – The composition about to be turned into structures.
Raises:: StageInputError – When the file is missing, empty, or does not satisfy the schema.
Return type:: None

static composition_of(item)[source]#

Read and check the item’s composition.json.

The same rules are enforced by the runner, which must stay debuggable standalone; checking here refuses a bad item before any job is built or submitted.

Parameters:: item (WorkItem) – The item whose directory holds the file.
Returns:: The decoded composition.
Return type:: dict
Raises:: StageInputError – Naming exactly what is wrong: unreadable or malformed JSON, a missing required key, a bad formula (it names child directories, so it must be alphanumeric), or elements and stoichiometry that do not line up.

build_jobspec(item)[source]#

Build the portable job description for one item.

Parameters:: item (WorkItem) – The item to build a job for. Its inputs have already passed validate_inputs.
Returns:: A portable job description any backend can execute.
Return type:: JobSpec
Raises:: StageError – If the stage is misconfigured such that no job can be built at all – a whole-stage fault, never a per-item verdict.

interpret_result(item, result)[source]#

Judge a finished structure job.

Parameters:

item (WorkItem) – The composition whose job finished.
result (JobResult) – What the backend reported.

Returns:

ok on exit 0, which includes a fan-out into zero children. Anything else is a fault carrying the runner’s stderr tail, and never terminal: a structure failure is an environment problem – missing pyxtal, a broken interpreter – that a retry can genuinely resolve. Per-space-group science rejections are absorbed inside the runner’s exit 0 and counted in its report.

Return type:

StageVerdict

The screen stage: one candidate structure -> keep (RELAXED) or drop (3.3).

The thin Stage half of the pair; the science lives entirely in screen_runner.py (self-contained: stdlib + mace/ase, zero matmelt imports), which this class ships to the backend as an input artifact of every job. This module itself imports no science – it builds job descriptions and judges results, nothing more, so it loads on any machine.

The runner MACE vc-relaxes the item’s structure.cif and writes a verdict screen_result.json (category RELAXED or FAILED + reasons + metrics); on RELAXED it also writes relaxed.cif. 1:1 – this stage is NOT fan-out: the item either advances or is screened out.

This is the funnel’s first terminal science verdict. A parsed FAILED verdict is a deterministic rejection – re-running the same relaxation with the same model and settings reproduces it – so interpret_result returns terminal=True: the item fails on the spot and spends no retry budget (G-2 verdict-vs-fault). Everything else (non-zero exit, missing/corrupt verdict) stays a retryable fault: missing mace is an environment problem, and a crashed relaxation could be an OOM – when in doubt, fault.

Model path, device, dtype and the relaxation knobs come from config via the constructor – never hardcoded (the 3.6 preflight/auto-download offer hooks into the same model_path option).

matmelt.core.stages.screen.RUNNER_NAME = 'screen_runner.py'#: The self-contained script shipped with every screen job. Lives next to this module inside the installed package; staged into the job’s cwd by the backend, so the command references it by bare name.

matmelt.core.stages.screen.STRUCTURE_NAME = 'structure.cif'#: The input file a structure child carries – declared as an input artifact so the Phase-4 staging transport knows to push it (it already sits in the item dir locally; backends stage in-place inputs as a no-op).

matmelt.core.stages.screen.VERDICT_NAME = 'screen_result.json'#: The verdict file the runner always writes on a completed screening.

matmelt.core.stages.screen.RELAXED_NAME = 'relaxed.cif'#: The relaxed structure the runner writes on a RELAXED verdict.

class matmelt.core.stages.screen.ScreenStage(python='python3', model_path='', model_size='medium', device='cpu', dtype='float64', fmax=0.05, max_steps=200)[source]#

Bases: Stage

MACE vc-relax one candidate structure and keep or screen it out.

Parameters:

python (str, default: 'python3') – Interpreter to invoke on the backend this stage is wired to (the user’s own environment holds the science stack – never bundled).
model_path (str, default: '') – Local MACE model file. Empty (default) = the MACE-MP foundation model, which mace downloads to its cache on first use – the runner discloses that before loading (3.6 adds the up-front consent offer).
model_size (str, default: 'medium') – Foundation model size when model_path is empty.
device (str, default: 'cpu') – Compute device passed to MACE (“cpu” | “cuda” | “mps” | “auto”).
dtype (str, default: 'float64') – Torch dtype; float64 (default) is required for a reliable stress tensor in vc-relax.
fmax (float, default: 0.05) – Force convergence target for the relaxation (eV/A).
max_steps (int, default: 200) – Maximum optimizer steps for the relaxation.

Raises:

StageError – For any invalid option, so a config mistake surfaces when the run is wired rather than as N failed jobs later.

Notes

The physical-sanity bounds that decide a verdict – bond lengths, stress, volume change, the energy window – are deliberately not options. They live as constants in the runner so verdicts stay comparable across a whole database.

runtime_requirements: tuple[Requirement, ...] = (Requirement(module='mace', pip_name='mace-torch', purpose='MACE force-field vc-relaxation (screen runner; brings ase and torch)'),)#: The science packages this stage’s runner imports, checked by the preflight before any work starts. Class attribute – a science stage overrides it; the default (nothing beyond stdlib) is itself a contract, e.g. extract’s.

property name: str#

Stable stage identifier – also the per-item idempotency key in state.

Returns:: The canonical name ("compose", "screen", …). It is what a config’s stages: block names, what the stage registry maps, and what the state store’s completion ledger records per item, so completing a stage twice is a no-op.
Return type:: str

property requires: StageInputSpec#

What must be true of an item’s directory before this stage runs.

Returns:: The declared input contract, enforced by validate_inputs.
Return type:: StageInputSpec

property produces: StageOutputSpec#

What this stage guarantees an item’s directory contains on success.

Returns:: The declared output contract, enforced by validate_outputs.
Return type:: StageOutputSpec

build_jobspec(item)[source]#

Build the portable job description for one item.

Parameters:: item (WorkItem) – The item to build a job for. Its inputs have already passed validate_inputs.
Returns:: A portable job description any backend can execute.
Return type:: JobSpec
Raises:: StageError – If the stage is misconfigured such that no job can be built at all – a whole-stage fault, never a per-item verdict.

interpret_result(item, result)[source]#

Judge the finished screening from its exit code and verdict file.

The funnel’s first terminal science verdict.

Parameters:

item (WorkItem) – The structure whose job finished; its directory holds the verdict file.
result (JobResult) – What the backend reported.

Returns:

One of four outcomes:

A non-zero or unknown exit is a retryable fault – missing mace, an out-of-memory kill, a broken interpreter.
Exit 0 with a RELAXED verdict advances the item.
Exit 0 with a FAILED verdict is a terminal science rejection: the same relaxation reproduces it, so no retry budget is spent and the recorded reason carries the failed checks.
Exit 0 with a missing, corrupt, or unknown verdict means the runner broke its contract. That is a fault, not a rejection – on unknown ground, let the budget apply.

Return type:

StageVerdict

The calculate stage: one screened-in structure -> DFTB+ result via the user’s AiiDA (6.5).

The thin Stage half of the pair; everything AiiDA lives entirely in calculate_runner.py (self-contained: stdlib + aiida/aiida_mydftbp/ase, zero matmelt imports), which this class ships to the backend as an input artifact of every job. This module itself imports no science and no AiiDA – it builds job descriptions and judges results, nothing more, so it loads on any machine.

The runner submits one mydftbp.single WorkChain per material through the user’s own AiiDA installation (aiida-core + the aiida-mydftbp plugin, whose WorkChain does the self-healing retries itself – exit codes 300-370, SCC/geometry/eigenvalue healing ladders, warm starts). Our package is the launcher/extractor around it and never ships or imports AiiDA at package level (the clean-room amendment: these are declared user-owned runtime requirements, exactly the smact/pyxtal/mace model). 1:1, optional, and skippable – configs without a calculate: entry behave exactly as before.

The runner writes calc_result.json (converged flag, WorkChain exit status, every healing attempt with SKF-stripped parameters). Verdicts follow the G-2 doctrine: a WorkChain that terminated with a healing-exhausted or configuration-fatal exit status (310, 320, 330, 340, 350, 360) is a terminal science verdict – the plugin already retried everything it knows, so re-running reproduces it. Transient statuses (300, 370), unknown statuses, and a broken/absent result file all stay retryable faults – when in doubt, fault.

validate_inputs refuses clearly when screen_result.json does not say RELAXED (extract’s precedent): calculate serves only screened-in materials, and a mid-pipeline start on a screened-out item is told exactly why before any job runs.

Every option comes from config via the constructor – code label, AiiDA profile, SKF locations, k-mesh, walltime, queue – never hardcoded.

matmelt.core.stages.calculate.RUNNER_NAME = 'calculate_runner.py'#: The self-contained script shipped with every calculate job. Lives next to this module inside the installed package; staged into the job’s cwd by the backend, so the command references it by bare name.

matmelt.core.stages.calculate.RELAXED_NAME = 'relaxed.cif'#: Inputs from the screen stage – declared as input artifacts so the staging transport knows to push them (in-place locally; staged as a no-op).

matmelt.core.stages.calculate.RESULT_NAME = 'calc_result.json'#: The result document this stage produces (attempt history included).

class matmelt.core.stages.calculate.CalculateStage(code_label='', profile=None, skf_label='ptbp_complete_set', skf_prefix='', skf_local_path=None, kpoints=None, max_wallclock_seconds=3600, queue_name=None, poll_interval_s=30.0, python='python3')[source]#

Bases: Stage

Run one screened-in material through DFTB+ via the user’s own AiiDA.

Parameters:

code_label (str, default: '') – Required. Label of the user’s configured AiiDA code for DFTB+ (e.g. "dftbplus@cluster") – verdi code list shows them.
profile (str | None, default: None) – AiiDA profile to load; None (default) uses the user’s default profile.
skf_label (str, default: 'ptbp_complete_set') – Label of the stored Slater-Koster FolderData node; ensured automatically by the runner (find-or-create, loud failure).
skf_prefix (str, default: '') – Required. Absolute directory of the .skf parameter files on the machine where DFTB+ actually executes – baked into the HSD input the WorkChain runs.
skf_local_path (str | None, default: None) – Local directory of .skf files used to create the labelled FolderData node when no node with skf_label exists yet. None (default) means find-only: a missing node is a loud, actionable failure, never a silent stall.
kpoints (list[int] | None, default: None) – SupercellFolding k-mesh diagonal, exactly three integers >= 1.
max_wallclock_seconds (int, default: 3600) – Walltime request per WorkChain attempt, in seconds.
queue_name (str | None, default: None) – Scheduler queue/partition for the WorkChain’s jobs; None (default) uses the code’s computer default.
poll_interval_s (float, default: 30.0) – Seconds between WorkChain status polls by the runner.
python (str, default: 'python3') – Interpreter to invoke on the backend this stage is wired to – the user’s AiiDA environment (their venv holds aiida-core, aiida-mydftbp and ase; never bundled).

Raises:

StageError – For any invalid option. The two required ones, code_label and skf_prefix, get messages naming verdi code list and the remote SKF directory respectively, so the fix is actionable at wiring time rather than after N failed WorkChains.

runtime_requirements: tuple[Requirement, ...] = (Requirement(module='aiida', pip_name='aiida-core', purpose='AiiDA engine the calculate runner submits WorkChains through'), Requirement(module='aiida_mydftbp', pip_name='aiida-mydftbp', purpose='DFTB+ plugin providing the self-healing mydftbp.single WorkChain'), Requirement(module='ase', pip_name='ase', purpose='CIF -> DFTB+ .gen geometry conversion in the calculate runner'))#: The science packages this stage’s runner imports, checked by the preflight before any work starts. Class attribute – a science stage overrides it; the default (nothing beyond stdlib) is itself a contract, e.g. extract’s.

property name: str#

Stable stage identifier – also the per-item idempotency key in state.

Returns:: The canonical name ("compose", "screen", …). It is what a config’s stages: block names, what the stage registry maps, and what the state store’s completion ledger records per item, so completing a stage twice is a no-op.
Return type:: str

property requires: StageInputSpec#

What must be true of an item’s directory before this stage runs.

Returns:: The declared input contract, enforced by validate_inputs.
Return type:: StageInputSpec

property produces: StageOutputSpec#

What this stage guarantees an item’s directory contains on success.

Returns:: The declared output contract, enforced by validate_outputs.
Return type:: StageOutputSpec

validate_inputs(item)[source]#

Refuse clearly unless the screening verdict actually says RELAXED.

Extends the base file-presence check by reading the verdict: DFTB+ time is expensive, and calculate serves only screened-in materials.

Parameters:: item (WorkItem) – The material about to be calculated.
Raises:: StageInputError – When a required file is missing or empty, when the screening verdict is unreadable or not valid JSON, or when it names any category other than RELAXED.
Return type:: None

build_jobspec(item)[source]#

Build the portable job description for one item.

Parameters:: item (WorkItem) – The item to build a job for. Its inputs have already passed validate_inputs.
Returns:: A portable job description any backend can execute.
Return type:: JobSpec
Raises:: StageError – If the stage is misconfigured such that no job can be built at all – a whole-stage fault, never a per-item verdict.

interpret_result(item, result)[source]#

Judge the finished calculation from its exit code and result file.

Parameters:

item (WorkItem) – The material whose WorkChain finished; its directory holds the result document.
result (JobResult) – What the backend reported for the runner process – note this is the runner’s exit code, not the WorkChain’s, which lives inside the result document.

Returns:

One of four outcomes:

A non-zero or unknown runner exit is a retryable fault – missing aiida, a stopped daemon, an unloadable profile.
Exit 0 with converged: true advances the item.
Exit 0 with a terminal WorkChain exit status (310, 320, 330, 340, 350, 360) is a terminal science verdict: the plugin’s own self-healing already retried everything it knows, or the failure is configuration-fatal, so re-running reproduces it.
Exit 0 with a transient status (300, 370), an unknown status, or a missing or corrupt result document is a fault – on unknown ground, let the budget apply.

Return type:

StageVerdict

The extract stage: one screened-in structure -> OPTIMADE-shaped record (3.4).

The thin Stage half of the pair; the logic lives entirely in extract_runner.py – which, unlike the other runners, is stdlib only: extraction is bookkeeping, not science, so it runs in any python anywhere with nothing to install (there is no exit-3 path).

1:1 and the funnel’s terminal stage: the runner reads the item’s relaxed.cif (parsed by its own minimal CIF reader – geometry is best-effort, a parse failure degrades to a geometry-less record, never a crash), screen_result.json (must say RELAXED: extract serves only screened-in materials), the optional structure.json provenance from the structure stage and the optional calc_result.json from the calculate stage (folded under _mcli.calculate – healing-attempt history included), and writes record.json – an OPTIMADE structures-shaped document ready for a database or an OPTIMADE-serving layer, with the non-standard extras namespaced under _mcli.

validate_inputs refuses clearly when screen_result.json does not parse to a RELAXED verdict – someone starting mid-pipeline on a screened-OUT (or hand-made) item is told exactly why, before any job runs.

matmelt.core.stages.extract.RUNNER_NAME = 'extract_runner.py'#: The self-contained (stdlib-only) script shipped with every extract job.

matmelt.core.stages.extract.RELAXED_NAME = 'relaxed.cif'#: Inputs from the screen stage – declared as input artifacts so the Phase-4 staging transport knows to push them (in-place locally; staged as a no-op).

matmelt.core.stages.extract.META_NAME = 'structure.json'#: Optional provenance from the structure stage (space group, lattice type); folded into the record when present, never required.

matmelt.core.stages.extract.CALC_NAME = 'calc_result.json'#: Optional result from the calculate stage (DFTB+ WorkChain outcome with the full healing-attempt history); folded under _mcli.calculate when present, never required – extract stays startable without calculate.

matmelt.core.stages.extract.RECORD_NAME = 'record.json'#: The OPTIMADE-shaped document this stage produces.

class matmelt.core.stages.extract.ExtractStage(python='python3')[source]#

Bases: Stage

Assemble one screened-in material’s OPTIMADE-shaped record.json.

Parameters:: python (str, default: 'python3') – Interpreter to invoke on the backend this stage is wired to. The runner needs nothing beyond the standard library, so any python works – the option exists for backend-environment symmetry with the other stages, never because a dependency demands it.
Raises:: StageError – When python is not a non-empty string.

property name: str#

Stable stage identifier – also the per-item idempotency key in state.

Returns:: The canonical name ("compose", "screen", …). It is what a config’s stages: block names, what the stage registry maps, and what the state store’s completion ledger records per item, so completing a stage twice is a no-op.
Return type:: str

property requires: StageInputSpec#

What must be true of an item’s directory before this stage runs.

Returns:: The declared input contract, enforced by validate_inputs.
Return type:: StageInputSpec

property produces: StageOutputSpec#

What this stage guarantees an item’s directory contains on success.

Returns:: The declared output contract, enforced by validate_outputs.
Return type:: StageOutputSpec

validate_inputs(item)[source]#

Refuse clearly unless the screening verdict actually says RELAXED.

Extends the base file-presence check by reading the verdict, which is the mid-pipeline-start guard: extract serves only screened-in materials.

Parameters:: item (WorkItem) – The material about to be extracted.
Raises:: StageInputError – When a required file is missing or empty, when the screening verdict is unreadable or not valid JSON, or when it names any category other than RELAXED.
Return type:: None

build_jobspec(item)[source]#

Build the portable job description for one item.

Parameters:: item (WorkItem) – The item to build a job for. Its inputs have already passed validate_inputs.
Returns:: A portable job description any backend can execute.
Return type:: JobSpec
Raises:: StageError – If the stage is misconfigured such that no job can be built at all – a whole-stage fault, never a per-item verdict.

interpret_result(item, result)[source]#

Judge a finished extract job.

Parameters:

item (WorkItem) – The material whose job finished.
result (JobResult) – What the backend reported.

Returns:

ok on exit 0, meaning the record was written. Anything else is a fault carrying the runner’s stderr tail, and never terminal: with a stdlib-only runner a failure is a broken input file or an environment accident, and the item already carries a RELAXED verdict – science never rejects here.

Return type:

StageVerdict

Built-in stages#

This Page