Built-in Stages

Built-in pipeline stages are generic, reusable stages implemented in core. They use type: instead of tool: and have fixed output artifact names.

Output Names

Stage type	Output artifacts
filter.tabular	filtered_table
filter.molecules	molecules
poses.extract	manifest, poses
molecules.hydrogenate	manifest, molecules
results.collect	table

Use these names in downstream references, for example rank_top.filtered_table or collect_results.table.

filter.tabular

Filters, derives columns, sorts, keeps top rows, and supports group-wise selection for CSV-like tables.

Input source—configure exactly one:

• table: upstream tabular artifact, usually a CSV/TSV score table or collected metrics table.
• config.input_path: CSV/TSV source file, useful when filter.tabular is the first pipeline stage. Relative paths resolve against the pipeline specification directory.

Optional inputs:

• ids: CSV artifact containing identifiers to retain before filtering and ranking. By default, filter.tabular matches the ligand column in both tables; override this with id_column for the main table and ids_column for the identifier table.

Output:

• filtered_table: CSV table written to output_name.

Common config keys:

• output_name: output CSV filename, default filtered_scores.csv.
• input_delimiter, output_delimiter: optional explicit delimiters; otherwise CSV/TSV selection follows the file suffix.
• derive: mapping of new column names to restricted expressions.
• filters: list of structured filters with column, op, and value.
• where: restricted boolean expression evaluated per row.
• sort_by: column used for sorting.
• ascending: sort direction, default true.
• group_by: column used for group-wise selection.
• top_n_per_group: keep this many rows per group after sorting.
• top_n: keep this many rows globally after filtering/sorting/grouping.
• columns: optional output column order/subset.
• id_column: column in table used with the optional ids input; default ligand.
• ids_column: column in ids used with the optional ids input; defaults to id_column.

Structured filter example:

- id: favorable_scores
  type: filter.tabular
  inputs:
    table: docking.scores
  config:
    filters:
      - column: best_score
        op: "<="
        value: -8.0
      - column: ligand
        op: "!="
        value: control
    sort_by: best_score
    ascending: true
    top_n: 500

Derived-column and expression example:

- id: efficient_ligands
  type: filter.tabular
  inputs:
    table: docking.scores
  config:
    derive:
      ligand_efficiency: best_score / heavy_atom_count
    where: ligand_efficiency <= -0.3 and heavy_atom_count > 12
    sort_by: ligand_efficiency
    ascending: true

Group-wise pose selection example:

- id: best_pose_per_ligand
  type: filter.tabular
  inputs:
    table: collected_pose_metrics.table
  config:
    group_by: ligand
    sort_by: comp_score
    ascending: true
    top_n_per_group: 1

Use group_by for ligand-level selection from pose-level tables. Global top_n alone selects the best rows across all ligands, not one pose per ligand.

Identifier-retention example:

- id: retained_poses
  type: filter.tabular
  inputs:
    table: docking.flexible_complexes_manifest
    ids: retained_ligands.filtered_table
  config:
    id_column: ligand
    ids_column: ligand
    group_by: ligand
    sort_by: CNNscore
    ascending: false
    top_n_per_group: 4

Source-stage example:

- id: initial_filter
  type: filter.tabular
  config:
    input_path: data/measurements.tsv
    filters:
      - column: status
        op: "=="
        value: accepted
    output_name: accepted.tsv

filter.molecules

Retains molecule or pose files based on a filtered table.

Required inputs:

• molecules: SDF file, directory, or path collection.
• ids: table containing IDs or exact paths to retain.

Output:

• molecules: retained SDF or retained path collection.

Common config keys:

• id_column: identifier column, default ligand_id.
• path_column: exact path column to retain, if available.
• output_name: output SDF filename, default filtered_molecules.sdf.

Preferred exact-path usage:

- id: retain_best_poses
  type: filter.molecules
  inputs:
    molecules: hydrogenated_poses.molecules
    ids: best_pose_per_ligand.filtered_table
  config:
    path_column: output_file

ID-based usage:

- id: retain_ligands
  type: filter.molecules
  inputs:
    molecules: docking.poses
    ids: top_scores.filtered_table
  config:
    id_column: ligand

Path columns are preferred because they avoid filename guessing. Auto-detected path columns include pose_file, molecule_file, ligand_file, path, and file.

poses.extract

Splits SDF/SDFGZ pose artifacts into one normalized SDF file per pose and writes a manifest.

Required inputs:

• poses: SDF/SDFGZ file, directory, or path collection.

Optional inputs:

• table: score/provenance table to join into the pose manifest.

Outputs:

• manifest: CSV with pose provenance.
• poses: path collection of one-pose SDF files.

Common config keys:

• output_name: manifest filename, default pose_manifest.csv.
• ligand_column: ligand ID column in the optional table, default ligand.
• path_column: path column in the optional table; auto-detected when omitted.
• score_column: optional SDF property to copy into the manifest.
• sort_by, ascending: optional manifest sorting.
• top_n_per_ligand: optional pose thinning after sorting.

Example:

- id: extract_poses
  type: poses.extract
  inputs:
    poses: gnina.poses
    table: gnina.scores
  config:
    output_name: pose_manifest.csv
    ligand_column: ligand
    path_column: pose_file

The manifest includes stable columns such as ligand, pose_id, pose_rank, pose_file, source_pose_file, and source_record_index.

molecules.hydrogenate

Repairs missing explicit hydrogens after docking and writes one repaired SDF per input molecule or pose.

Required inputs:

• molecules: SDF file, directory, or path collection.

Optional inputs:

• table: metadata table to preserve ligand and pose IDs.
• templates: SDF file, directory, or path collection containing reference molecules for template-based bond-order repair.

Outputs:

• manifest: CSV with repair provenance.
• molecules: path collection of repaired SDF files.

Common config keys:

• ligand_column: ligand ID column, default ligand.
• pose_id_column: pose ID column, default pose_id.
• template_path_column: explicit template path column, auto, or empty. Default auto.
• on_template_failure: warn, fail, or ignore. Default warn.
• on_failure: fail or warn. Default fail.
• add_coords: add coordinates for hydrogens, default true.
• output_name: manifest filename, default hydrogenated_manifest.csv.

Direct hydrogen addition:

- id: hydrogenate_poses
  type: molecules.hydrogenate
  inputs:
    molecules: extract_poses.poses
    table: extract_poses.manifest
  config:
    ligand_column: ligand
    pose_id_column: pose_id
    on_failure: fail

Template-based repair:

- id: hydrogenate_poses
  type: molecules.hydrogenate
  inputs:
    molecules: extract_poses.poses
    table: extract_poses.manifest
    templates: prepared_ligands.molecules
  config:
    template_path_column: auto
    on_template_failure: warn

Template-based repair is preferred when available because it can preserve protonation state and bond orders from prepared ligand inputs. The manifest records template_file, template_status, hydrogens_before, hydrogens_after, status, and message.

results.collect

Collects JSON result files from scattered jobs, extracts configured JSON-path metrics, and joins scatter or table provenance.

Required inputs:

• results: result file, result path collection, or reduced scattered result artifact.

Optional inputs:

• table: provenance table. If omitted, results.collect uses scatter provenance when present.

Output:

• table: CSV table written to output_name.

Common config keys:

• output_name: output CSV filename, default collected_results.csv.
• metrics: mapping of output column names to JSON paths.

Example:

- id: collect_openbpmd
  type: results.collect
  inputs:
    results: openbpmd_per_pose.result_record
  config:
    metrics:
      comp_score: metrics.comp_score
      pose_score: metrics.pose_score
    output_name: openbpmd_pose_metrics.csv

If metrics is omitted, the stage collects status. JSON paths use dot notation through nested objects and lists, for example metrics.delta_g_kcal_mol or extra.molecule_results.0.name.

Pose-Level Pattern

For pose-level pipelines:

1. Filter ligands or poses from docking tables.
2. Use poses.extract to create one file per pose.
3. Use molecules.hydrogenate before MD or free-energy stages that need explicit hydrogens.
4. Scatter pose-level tools over the repaired path collection.
5. Use results.collect to collect per-pose metrics.
6. Use filter.tabular with group_by: ligand and top_n_per_group to select poses per ligand.
7. Use filter.molecules with an exact path column to retain the selected files.