Backend contract#

The ComputeBackend ABC — the where half — plus the backend registry.

The ComputeBackend contract: six methods every execution target implements.

Interface only – no subprocess, no SSH, nothing executable lives here. The concrete backends plug in behind it: local (subprocess, Phase 1.4) and ssh_exec (SSH / SLURM, Phase 4), looked up by kind through registry.py. Keeping the interface free of any execution detail is the “separate what from where” thesis applied to the compute plane: a stage builds a portable JobSpec and never learns which machine ran it; making the screening stage run on a remote GPU box is a config change, not a code change.

Adding a backend is one new subclass plus one register(kind, cls) call – no stage, orchestrator, or CLI change. If adding a backend seems to require touching a stage, the abstraction is leaking: stop.

class matmelt.backends.base.ComputeBackend(name, options)[source]#

Bases: ABC

One uniform interface over every execution target.

Lifecycle contract – the order a driver calls these in:

capabilities() / health_check()   # before any work is handed over
handle = submit(spec)             # non-blocking: work starts, handle returned
while not poll(handle).is_terminal:
    ...
result = fetch(handle)            # faithful exit code + streams + outputs
release(handle)                   # idempotent cleanup, always (finally)

Invariants every implementation must hold (do not weaken):

  • Two failure tiers. A job that runs but exits non-zero is a normal JobResult with a non-zero exit_code – NOT an exception. Only infrastructure failures (unreachable target, unknown handle, malformed spec) raise BackendError. Judging scientific success is the stage’s job; the backend only reports faithfully what happened.

  • Handles are durable. A Handle must keep resolving after the tool restarts (native id + persisted paths, never a live object), so an interrupted matmelt run re-polls its in-flight jobs instead of orphaning or double-submitting them.

  • ``health_check`` never raises. Being down is the normal answer it reports.

  • No hardcoded compute details. Hosts, paths, queues, credentials come from the options dict a backend is constructed with (config / env:NAME references) – never a string literal in backend code.

abstract capabilities()[source]#

Report what this backend offers (cpu/gpu/slurm, max_parallel).

Pure introspection – nothing is submitted, nothing mutates. A stage’s Resources needs are asserted against this before any work is handed over, so an impossible placement (a GPU stage on a CPU-only backend) is caught up front rather than as a late job failure.

Returns:

What this target offers. max_parallel is advertised here and enforced by the orchestrator, not the backend.

Return type:

Capabilities

abstract submit(spec)[source]#

Start the work described by spec; return a durable Handle immediately.

Non-blocking: stages spec.input_artifacts as the target requires (in place locally, uploaded remotely), launches the command, and returns – progress is observed through poll, never by waiting here.

Parameters:

spec (JobSpec) – The portable job description built by a stage.

Returns:

Durable identity for the started job, safe to persist and re-resolve after a restart.

Return type:

Handle

Raises:

BackendError – Only when the backend cannot accept the job: unreachable target, malformed spec, an input it cannot stage. A job that starts and then fails is reported through poll / fetch, never raised here.

abstract poll(handle)[source]#

Return the current JobStatus of the job behind handle.

Cheap and idempotent: called repeatedly until terminal, and calling it again after that keeps returning the same terminal status. Must work from the handle’s persisted identity alone (no in-memory job map required), so a restarted tool can poll jobs it did not launch.

Parameters:

handle (Handle) – The handle submit returned, possibly reloaded from the state store’s in-flight ledger in a later process.

Returns:

Where the job is now.

Return type:

JobStatus

Raises:

BackendError – When the handle is unknown to this backend, or the status genuinely cannot be obtained.

abstract fetch(handle)[source]#

Collect a finished job’s result: exit code, streams, output artifacts.

Meaningful once poll reports a terminal status.

Parameters:

handle (Handle) – The handle of a job that has reached a terminal status.

Returns:

The faithful outcome. exit_code is reported, never raised: non-zero is returned as-is, and None means “ran, code unknown” and must never be defaulted to 0. output_artifacts holds the files named in JobSpec.output_artifacts that were actually found – a missing one is absent from the dict, for the stage to judge.

Return type:

JobResult

Raises:

BackendError – Only when the result itself cannot be retrieved: unknown handle, lost job directory, unreachable target.

abstract health_check()[source]#

Report whether this backend can take work right now. Never raises.

The driver checks health before handing a backend any work and simply skips a sick backend until it recovers.

Returns:

Being down is the normal answer: HealthStatus(ok=False, detail=...) with a short, human-readable, secret-free reason for logs and doctor output.

Return type:

HealthStatus

abstract release(handle)[source]#

Clean up whatever the backend holds for handle. Idempotent.

Called by the driver of the job lifecycle in a finally block (after fetch, or when abandoning a job): remove scratch dirs, kill a still-live process, forget bookkeeping. Never judges or records the job’s result; that is the caller’s job.

Parameters:

handle (Handle) – The handle to clean up. Releasing an unknown or already-released handle is a no-op, not an error – restart-time cleanup must always be safe.

Return type:

None

The backend registry: kind string -> ComputeBackend class.

The single seam where config becomes a running backend. A user’s YAML names each backend only by kind (local, ssh_exec, …); this module maps that string to the concrete class and instantiates it. Because the lookup is by string, adding a backend is one subclass + one register call – no stage, orchestrator, or CLI change.

Registration is an explicit call, not an import-time decorator, so there is no hidden import-order dependency: a concrete backend module calls register("local", LocalBackend) at the bottom of its file, and whoever wires the tool imports the backend modules it intends to offer.

Construction contract: a registered class is instantiated as cls(name, options) – the backend’s configured name plus its free-form options dict. The dict (rather than unpacked kwargs) keeps the contract stable as backends grow new options; each backend validates what it needs in its own __init__ and raises BackendError for a missing/invalid option.

matmelt.backends.registry.register(kind, cls)[source]#

Register cls as the implementation of kind.

Called explicitly at the bottom of each concrete backend module.

Parameters:

  • kind (str) – The string a config’s backends.<name>.kind names.

  • cls (type[ComputeBackend]) – The implementing class, which must be a ComputeBackend subclass.

Raises:

BackendError – If cls is not a ComputeBackend subclass, or if kind is already registered to a different class. Re-registering the same class is idempotent, so a module re-import is a no-op, but silently shadowing an existing kind is refused.

Return type:

None

matmelt.backends.registry.get(kind)[source]#

Return the class registered under kind.

Parameters:

kind (str) – The backend kind to look up.

Returns:

The registered ComputeBackend subclass.

Return type:

type

Raises:

BackendError – If nothing is registered under kind. The message names the kinds that are known, so a config typo produces an actionable message rather than a KeyError.

matmelt.backends.registry.create_backend(name, kind, options)[source]#

Instantiate the backend name of kind with its options dict.

The one seam where a config block becomes a live backend.

Parameters:

  • name (str) – The user’s configured name for this instance, carried into every Handle it issues.

  • kind (str) – Which registered implementation to construct.

  • options (dict[str, Any]) – The backend’s free-form options: block, passed through per the cls(name, options) contract.

Returns:

The constructed, option-validated backend.

Return type:

ComputeBackend

Raises:

BackendError – If kind is unknown, or the backend’s own __init__ validation rejects an option – a missing required value or a bad one.

Examples

>>> from matmelt.backends import local  # registers kind "local"
>>> backend = create_backend("laptop", "local", {"max_parallel": 4})
>>> backend.name, backend.capabilities().max_parallel
('laptop', 4)
matmelt.backends.registry.known_kinds()[source]#

List the backend kinds currently registered.

Returns:

Sorted kinds, used in diagnostics and error messages. Which kinds are present depends on which backend modules have been imported.

Return type:

list of str