ML Plugin¶

Package: caddforge-ml

Public source tag

This page describes caddforge-ml 0.4.0, tagged as v0.4.0. Published on PyPI; see the version matrix.

The ML plugin exposes model training, prediction, dimensionality reduction, model sweeps, and classifier threshold exploration workflows.

Requirements and example status¶

The example library provides downloadable configuration templates for every workflow on this page, plus a shared Python runner. File paths refer to inputs you must prepare; these are not bundled research datasets. Schema validation does not establish scientific validity or engine availability. See the version matrix for release tags and publication status.

`simple-gp`¶

Trains and predicts with a simple Gaussian process regression workflow.

CLI:

caddforge run ml simple-gp --config simple_gp.yaml

YAML:

train_path: data/train.csv
target_column: y
feature_columns: [f1, f2, f3]
smiles_column: smiles
ignore_columns: [id, comment]
fingerprint_types: [morgan, atompair, featmorgan]
kernel: tanimoto
noise: 0.05
test_path: data/test.csv
save_report: true
log_targets: false
generate_scatter_plot: true
scatter_filename: gp_scatter.html
mode: train
model_path: simple_gp_model.joblib
id_column: id

Python:

from caddforge.core.execution import load_result_for_job, submit_workflow
from caddforge.plugins import get_tool

spec = get_tool("ml", "simple-gp")
cfg = spec.config_cls(train_path="data/train.csv", target_column="y", test_path="data/test.csv")
workflow = spec.workflow_cls(cfg)
job = submit_workflow(
    workflow, cfg, backend_name=cfg.backend,
    input_ports=tuple(getattr(spec.plugin, "input_ports", ())),
)
result = load_result_for_job(job)
if result is not None:
    print(result.metrics)

Outputs to inspect: results/predictions.csv, results/results.json, results/result_record.json.

`pca`¶

Runs principal component analysis over configured feature columns.

CLI:

caddforge run ml pca --config pca.yaml

YAML:

data_path: data/features.csv
feature_columns: [f1, f2, f3]
ignore_columns: [id]
n_components: 5
standardize: true
drop_missing_rows: true
scores_filename: pca_scores.csv
components_filename: pca_components.csv
results_filename: results.json

Python:

from caddforge.core.execution import load_result_for_job, submit_workflow
from caddforge.plugins import get_tool

spec = get_tool("ml", "pca")
cfg = spec.config_cls(data_path="data/features.csv", n_components=5, backend="local")
workflow = spec.workflow_cls(cfg)
job = submit_workflow(
    workflow, cfg, backend_name=cfg.backend,
    input_ports=tuple(getattr(spec.plugin, "input_ports", ())),
)
result = load_result_for_job(job)
if result is not None:
    print(result.metrics)

Outputs to inspect: results/pca_scores.csv, results/pca_components.csv, results/results.json.

`model-zoo`¶

Runs a scikit-learn model from the model registry for regression or classification.

CLI:

caddforge run ml model-zoo --config model_zoo.yaml

YAML:

train_path: data/train.csv
target_column: y
feature_columns: [f1, f2]
model_name: random_forest
task: regression
test_size: 0.2
random_state: 123
standardize_targets: false
target_transform: none
positive_class: 1
predictions_filename: predictions.csv
results_filename: results.json
metrics_filename: metrics.json
mode: train
model_path: artifacts/model.joblib

Python:

from caddforge.core.execution import load_result_for_job, submit_workflow
from caddforge.plugins import get_tool

spec = get_tool("ml", "model-zoo")
cfg = spec.config_cls(
    train_path="data/train.csv",
    target_column="y",
    model_name="random_forest",
    task="regression",
    backend="local",
)
workflow = spec.workflow_cls(cfg)
job = submit_workflow(
    workflow, cfg, backend_name=cfg.backend,
    input_ports=tuple(getattr(spec.plugin, "input_ports", ())),
)
result = load_result_for_job(job)
if result is not None:
    print(result.metrics)

Outputs to inspect: results/predictions.csv, results/metrics.json, results/results.json, artifacts/model.joblib.

`model-factory`¶

Runs multiple model-zoo training jobs against the same data and aggregates metrics, uncertainty diagnostics, confusion summaries, and model artifacts. Predict-only execution is provided by model-zoo, not by the factory.

CLI:

caddforge run ml model-factory --config model_factory.yaml

YAML:

train_path: data/train.csv
target_column: y
task: regression
feature_columns: [f1, f2]
ignore_columns: [compound_id, measured_at]
test_path: null
smiles_column: smiles
fingerprint_types: [morgan, atompair, featmorgan]
fp_bits: 2048
fp_radius: 2
fp_use_chirality: true
model_names: [ridge, random_forest, xgboost]
model_params:
  random_forest:
    n_estimators: 500
    random_state: 123
calibrate: false
calibrate_method: sigmoid
test_size: 0.2
random_state: 123
predictions_filename: predictions.csv
results_filename: results.json
metrics_filename: metrics.json
id_column: compound_id
classification_thresholds: null
time_column: measured_at
standardize_targets: false
positive_class: null
target_transform: none
drop_non_positive_targets: true
# log_targets: false  # deprecated alias for target_transform: log

Set task explicitly when possible. With an explicit task and no model_names, the factory runs all compatible registry models. If both values are omitted, the factory attempts every registered model and each subrun infers its own task, so incompatible model/task combinations can be included.

Model-factory configuration¶

These are all options defined by ModelFactoryConfig. Shared options inherited from the core, including backend, run_directory, registry_path, job_name, result export, and Slurm settings, are described in Configuration.

Option	Type and default	Description
`train_path`	path, required	Training CSV containing the target and every configured training column.
`target_column`	string, required	Column to predict. Numeric values are required for regression and threshold-binned classification.
`task`	`regression`, `classification`, or `null`; default `null`	Task passed to every model-zoo subrun. If omitted, a subrun infers classification for thresholded or non-numeric targets and regression otherwise. An explicit value also filters automatic model selection.
`feature_columns`	list of strings or `null`; default `null`	Numeric features. When omitted, all columns except the target, SMILES column, and ignored columns are selected. They are concatenated with fingerprints when SMILES are also configured.
`ignore_columns`	list of strings; default `[]`	Excludes columns from automatic feature selection only. It does not alter explicit `feature_columns`.
`test_path`	path or `null`; default `null`	External evaluation CSV. When present, all training rows are fitted; otherwise the training CSV is split into fit and holdout sets.
`smiles_column`	string or `null`; default `null`	SMILES column from which RDKit fingerprints are generated. Fingerprints can supplement or replace tabular features.
`fingerprint_types`	list of strings; default `[morgan, atompair, featmorgan]`	Fingerprints concatenated per molecule. Supported values: `morgan`, `atompair`, and `featmorgan`. Used only when `smiles_column` is set.
`fp_bits`	integer; default `2048`	Number of bits per fingerprint type. Total fingerprint width is `fp_bits` multiplied by the number of types.
`fp_radius`	integer; default `2`	Radius for Morgan and feature-Morgan fingerprints.
`fp_use_chirality`	boolean; default `true`	Include chirality in fingerprint generation.
`model_names`	list of strings or `null`; default `null`	Registry models to run. Valid values are `ridge`, `lasso`, `elasticnet`, `bayesian_ridge`, `logistic_regression`, `random_forest`, `extra_trees`, `svm`, `knn`, `gaussian_process`, and `xgboost`. With an explicit task, `null` selects every compatible model.
`model_params`	mapping; default `{}`	Estimator keyword arguments keyed by model name. Each nested mapping is passed directly to the corresponding scikit-learn or XGBoost constructor.
`calibrate`	boolean; default `false`	For classification, calibrate probabilities with three-fold `CalibratedClassifierCV`. Ignored for regression.
`calibrate_method`	`sigmoid` or `isotonic`; default `sigmoid`	Probability calibration method used when calibration is enabled.
`test_size`	float from `0.0` to `0.9`; default `0.2`	Holdout fraction when `test_path` is absent; use a positive value when a holdout is required. Splitting is chronological with `time_column`; otherwise it is random and classification is stratified when possible.
`random_state`	integer; default `123`	Seed for the random holdout split. Estimator seeds must be set separately in `model_params`.
`predictions_filename`	string; default `predictions.csv`	Predictions filename in each model subrun's `results/` directory.
`results_filename`	string; default `results.json`	Model-zoo summary filename in each model subrun's `results/` directory.
`metrics_filename`	string; default `metrics.json`	Metrics filename in each model subrun's `results/` directory.
`id_column`	string or `null`; default `null`	Column copied to per-model predictions as `id`; row indices are used when omitted.
`classification_thresholds`	list of floats or `null`; default `null`	Sorted boundaries used with classification to bin a continuous target using `numpy.digitize`. N thresholds produce N+1 integer classes. Factory confusion summaries require a binary outcome.
`time_column`	string or `null`; default `null`	Datetime-like column for chronological splitting when no external test CSV is supplied. The newest holdout fraction is evaluated.
`standardize_targets`	boolean; default `false`	Regression only. Standardizes the target after any transform, then reverses standardization and transformation for predictions and metrics.
`positive_class`	integer or `null`; default `null`	Positive label for binary probability and confusion diagnostics. Subruns prefer class `1`, then the highest class, when omitted; factory confusion summaries default to `1`.
`log_targets`	boolean; default `false`	Deprecated alias for `target_transform: log`. It cannot be combined with a non-`none` target transform.
`target_transform`	`none`, `log`, or `log1p`; default `none`	Regression-only transform applied before optional standardization and inverted for outputs. `log` requires targets greater than zero; `log1p` requires targets greater than -1.
`drop_non_positive_targets`	boolean; default `true`	Drops targets invalid for the selected log transform. If false, invalid targets cause the run to fail.

Model compatibility:

Task	Models
Regression only	`ridge`, `lasso`, `elasticnet`, `bayesian_ridge`, `gaussian_process`
Classification only	`logistic_regression`
Regression and classification	`random_forest`, `extra_trees`, `svm`, `knn`, `xgboost`

Python:

from caddforge.core.execution import load_result_for_job, submit_workflow
from caddforge.plugins import get_tool

spec = get_tool("ml", "model-factory")
cfg = spec.config_cls(
    train_path="data/train.csv",
    target_column="y",
    task="regression",
    backend="local",
)
workflow = spec.workflow_cls(cfg)
job = submit_workflow(
    workflow, cfg, backend_name=cfg.backend,
    input_ports=tuple(getattr(spec.plugin, "input_ports", ())),
)
result = load_result_for_job(job)
if result is not None:
    print(result.metrics)

Outputs to inspect:

results/summary.csv and results/summary.json: one row per model with prefixed metrics, optional uncertainty-to-error correlation, and binary confusion values. Regression summaries also include OLS slope, intercept, R2, and RMSE diagnostics.
<model-name>/: complete model-zoo subruns with predictions, metrics, results, logs, and model bundles.
artifacts/<model-name>_model.joblib: pooled copies of generated model bundles.
results/result_record.json: canonical result exposing summary_csv, summary_json, and log paths.

`explore-classifier`¶

Launches an interactive Streamlit UI for classifier threshold exploration.

CLI:

caddforge run ml explore-classifier --config explore_classifier.yaml

YAML:

job_id: existing_model_job_id
run_base: runs
model_name: random_forest
positive_class: 1

Python:

from caddforge.core.execution import load_result_for_job, submit_workflow
from caddforge.plugins import get_tool

spec = get_tool("ml", "explore-classifier")
cfg = spec.config_cls(
    job_id="existing_model_job_id",
    run_base="runs",
    model_name="random_forest",
    backend="local",
)
workflow = spec.workflow_cls(cfg)
job = submit_workflow(
    workflow, cfg, backend_name=cfg.backend,
    input_ports=tuple(getattr(spec.plugin, "input_ports", ())),
)
result = load_result_for_job(job)
if result is not None:
    print(result.status.value)

Outputs to inspect: Streamlit UI session, saved threshold JSON under artifacts/, updated model artifact when possible.

Compressed inputs¶

Training, test, and PCA CSV inputs may be supplied as .csv.gz. They are expanded only at runtime, and saved configurations retain the compressed source paths.

ML Plugin¶

Requirements and example status¶

simple-gp¶

pca¶

model-zoo¶

model-factory¶

Model-factory configuration¶

explore-classifier¶

Compressed inputs¶

`simple-gp`¶

`pca`¶

`model-zoo`¶

`model-factory`¶

`explore-classifier`¶