Skip to content

Pipeline Specification

Pipeline YAML uses stages. A stage is either a tool plugin stage or a generic built-in stage.

version: 1
name: docking-screen

defaults:
  backend: local
  registry_path: runs/job_registry.sqlite

stages:
  - id: unidock_screen
    tool: docking/unidock
    config: unidock.yaml
    scatter:
      by: ligand_multisdf_path
      chunk_size: 50000
    outputs:
      scores: best_scores_csv
      poses: poses_dir

  - id: top_score_filter
    type: filter.tabular
    inputs:
      table: unidock_screen.scores
    config:
      sort_by: best_score
      ascending: true
      top_n: 1000

Rules

  • tool stages resolve installed plugins through the registry.
  • type stages call generic built-ins from core.
  • inputs reference upstream artifacts as stage_id.artifact_name.
  • outputs map pipeline artifact names to workflow result path keys. If omitted, CADDForge uses the tool plugin's declared output ports.
  • Optional plugin output ports are skipped when a workflow configuration does not produce the corresponding result path. Manifest-backed output ports can expose a CSV path column as a multi-path artifact for downstream scatter or filtering.
  • If a scattered stage receives additional path-collection inputs with the same length as the scatter input, those paths are bound by row order for each shard. Use this only for paired artifacts selected from the same provenance table, such as ligand SDF and protein PDB files for the same pose. For paired list[Path] config fields, each shard receives a one-item list with its matching path.
  • Scatter strategies include sdf, csv, tsv, lines, and auto. Plugin split contracts can apply these strategies to path members inside structured collection fields and declare deterministic output reducers.
  • Stage IDs are part of the artifact API and should remain stable in examples.
  • Defaults are applied centrally by the executor.