Configuration

The config schema, env: resolution, and the wiring from YAML to live objects.

The config schema: what a user’s YAML must look like (2.3).

Pydantic v2 models, shape only – no file I/O, no env: resolution, no backend construction (those are the loader’s job). Every model forbids unknown keys (extra="forbid"), so a typo like retry_budgett fails loudly with its YAML path instead of being silently ignored.

The shape mirrors the what-vs-where thesis:

run:        this run's identity and inputs (name, work_dir, elements)
pipeline:   how the funnel executes (stop_after, retry budget, timings)
backends:   name -> {kind, options}   -- the WHERE; free-form options
stages:     name -> {backend, options} -- the WHAT; names its backend

A stage references a backend only by name; the cross-check that the name is defined (and that stop_after names a real stage) lives on the root model, so an inconsistent file never validates. Values may arrive as strings (the env:NAME resolution always produces strings); pydantic’s lax mode coerces numeric strings into the typed fields, while still rejecting genuinely wrong types – including YAML booleans where numbers belong.

matmelt.config.schema.NonEmptyStr

A string that must not be empty (list items get no YAML path of their own, so the constraint carries the message).

alias of Annotated[str, StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=1, max_length=None, pattern=None, ascii_only=None)]

class matmelt.config.schema.RunCfg(**data)

Bases: _Model

This run’s identity and scientific inputs – the run: block.

name

A short label for the run, shown by status and results.

Type:

str

work_dir

Root of the run’s on-disk workspace. The state database and every per-material item directory live under it.

Type:

str

elements

The element set the compose stage enumerates over.

Type:

list of str

combination_sizes

How many elements each seed combines: [2] explores binaries, [2, 3] adds ternaries, and so on. Size 1 is legal and seeds elemental solids. Omitted – or explicitly [] – means every size from 1 to the number of distinct elements.

Enumeration grows combinatorially, so the preflight reports the seed count before any work starts, and heavy stages can be pointed at a remote backend when a laptop is not enough.

Type:

list of int

model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class matmelt.config.schema.PipelineCfg(**data)

Bases: _Model

How the funnel executes – the pipeline: block. Every key is optional.

stop_after

Name of the last stage this run executes, or None for the full configured funnel.

A mid-funnel stop is a pause, not a retirement: materials rest at that stage’s output state, live and claimable, so raising or removing stop_after – or running --only <stage> – continues them on the same store. Only a run that executes through the configured last stage retires its materials as first-class DONE.

Type:

str or None

retry_budget

Per-stage fault budget. A faulted material is requeued until the budget is spent, then goes terminal FAILED. Deterministic science verdicts bypass it entirely – no retry can change them.

Type:

int

lease_timeout_s

Age beyond which a claim is considered orphaned and released back to its queue.

Type:

float

pass_interval_s

Sleep between orchestrator passes.

Type:

float

model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class matmelt.config.schema.BackendCfg(**data)

Bases: _Model

One compute backend – an entry under backends:, and the where.

kind

Selects the implementation through the backend registry: local, ssh_exec, or anything else registered.

Type:

str

options

That kind’s own settings, deliberately free-form here because each kind needs different keys – local wants work_dir and max_parallel, ssh_exec wants host and friends. The backend’s own constructor validates what it needs and rejects what it cannot use, which keeps this schema stable as backends grow options.

Type:

dict

model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class matmelt.config.schema.StageCfg(**data)

Bases: _Model

One pipeline stage – an entry under stages:, and the what.

backend

The name of an entry under backends:. This is the only place a stage says where it runs, and it is one word of config rather than any code – moving a stage to a bigger machine is editing this line.

Type:

str

options

Stage-specific settings – a model path, a device, thresholds – unpacked onto the stage’s typed constructor signature, which validates them.

Type:

dict

model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class matmelt.config.schema.Config(**data)

Bases: _Model

The fully-validated configuration file – the tool’s single control surface.

run

Identity and scientific inputs.

Type:

RunCfg

pipeline

How the funnel executes.

Type:

PipelineCfg

backends

The compute targets, by name.

Type:

dict

stages

The funnel stages, by name.

Type:

dict

Notes

Beyond the per-field checks, the root model enforces the two cross-references an inconsistent file could break: every stages.<name>.backend must name a defined backend, and pipeline.stop_after, when set, must name a defined stage. Every model also forbids unknown keys, so a typo is an error rather than a setting that silently does nothing.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

Load and resolve the config file; wire it into a runtime (2.3).

The pipeline, each step failing as one ConfigError with YAML paths:

raw text --(yaml.safe_load)--> dict
         --(recursive "env:NAME" resolution from os.environ)--> dict
         --(pydantic validation, schema.py)--> Config
         --(build_runtime: the ONLY place config meets the backend
            registry)--> Runtime

env:NAME is how secrets and machine-specific values stay out of the file (and out of git): any string value anywhere in the YAML may be env:NAME, replaced by os.environ["NAME"] before validation. An unset variable is a hard error naming the variable and its YAML path – it never resolves to an empty string, because an empty string silently becoming a hostname or a password is exactly the failure mode this exists to prevent.

build_runtime turns the validated Config into live objects: it imports the built-in backend modules (registration is explicit-import based), asks the registry to construct each configured backend, and pre-computes the {stage_name: backend} mapping the orchestrator takes. Core never sees any of this – the dependency arrow points config -> backends, never backwards.

exception matmelt.config.loader.ConfigError

Bases: Exception

Any problem loading, resolving, or validating the config file.

Always carries an actionable message with the YAML path of the offending value (all validation problems are collected into one raise), so the user fixes the file, not a stack trace.

matmelt.config.loader.load_config(path)

Load the YAML file at path into a validated Config.

Parameters:

path (str | PathLike[str]) – The config file to read.

Returns:

The fully-validated configuration.

Return type:

Config

Raises:

ConfigError – For a missing, unreadable or empty file; invalid YAML; a non-mapping top level; an unset env: reference; or any schema violation. Every schema violation is reported together, one line each with its YAML path, so a user fixes the whole file in one pass rather than one error per run.

class matmelt.config.loader.Runtime(config, backends, stage_backends)

Bases: object

The validated config plus the live backends built from it.

config

The validated configuration.

Type:

Config

backends

One constructed ComputeBackend per entry under backends:, keyed by the user’s configured name.

Type:

dict

stage_backends

The same instances re-keyed by stage name – the exact {stage_name: backend} mapping the Orchestrator takes. Two stages naming the same backend share one instance, and therefore share its max_parallel slots.

Type:

dict

matmelt.config.loader.build_runtime(config)

Construct every configured backend – where config meets the registry.

Parameters:

config (Config) – The validated configuration.

Returns:

The config, the constructed backends, and the stage-to-backend mapping.

Return type:

Runtime

Raises:

ConfigError – For an unknown kind or an option a backend rejects, carrying the entry’s YAML path. At this point such a failure is a configuration problem, not an infrastructure one, which is why it is re-raised rather than left as a BackendError.

Notes

The built-in backend modules are imported here so their kinds register. Registration is an explicit call at the bottom of each module, never import-time magic, so there is no hidden ordering dependency.

matmelt.config.loader.build_stages(config)

Construct the configured stages in funnel order – config meets the registry.

Parameters:

config (Config) – The validated configuration.

Returns:

Exactly what the Orchestrator takes: each stage consumes the previous one’s output state.

Return type:

list of Stage

Raises:

ConfigError – For an unknown stage name; for a configured set that does not follow the funnel; or for an option a stage does not accept or rejects, carrying the entry’s YAML path.

Notes

The configured names must form an ordered subsequence of the funnel compose -> structure -> screen -> [calculate] -> extract, omitting only optional stages. Every required stage up to the furthest configured one must be present: a funnel starting mid-chain would never consume NEW, and a hole at a required stage is refused rather than silently bridged.

The YAML order of the stages: entries is cosmetic – it is a mapping. The funnel defines the order.