CADDForge ML Tool

Install this plugin with python -m pip install caddforge-ml. It depends directly on caddforge-core>=0.21.0; Python imports and CLI commands still use caddforge. For the complete public toolbox, including user-facing extras, install python -m pip install caddforge. When migrating an existing environment, uninstall the former caddforge core distribution before installing renamed core and updated plugins.

Machine learning workflows for the CADDForge toolbox. The initial tool provides a simple Gaussian Process regressor (Tanimoto or RBF kernel) with one-file training + inference for rapid prototyping. Additional tools include PCA and a scikit-learn model zoo.

Repo Layout

• src/cadd_ml/simple_gp/: core GP primitives and workflow.
• src/cadd_ml/pca/: PCA workflow.
• src/cadd_ml/model_zoo/: scikit-learn model zoo workflow.
• tests/: unit/integration tests.
• src/cadd_ml/model_factory/: orchestrates multiple model-zoo runs in one go.
• src/cadd_ml/explore_classifier/: interactive threshold exploration for classifiers.

Tools at a Glance

• simple-gp (regression): lightweight GP with optional PCA preprocessing, log-targets, standardization, and model serialization for predict-only runs.
• pca (preprocessing): standalone PCA with standardized feature handling and explained-variance reporting.
• model-zoo (regression/classification): unified runner for scikit-learn models (ridge, lasso, elasticnet, bayesian_ridge, logistic_regression, random_forest, extra_trees, svm, knn, gaussian_process, xgboost) with log/standardize targets, time split, and predict-only mode.
• model-factory (regression/classification): runs multiple model-zoo models, aggregates metrics/uncertainty correlations/confusion matrices, and pools artifacts under the main run.
• explore-classifier (classification UI): Streamlit UI to tweak decision thresholds, view live metrics/curves, plot metrics vs threshold, and save thresholds into model artifacts for downstream prediction runs.

Simple Gaussian Process Tool (regression)

With caddforge-core available (or PYTHONPATH pointing to src/), the simple-gp tool is exposed via the caddforge.tools entry point:

# List tools
caddforge list-tools

# Show template config
caddforge show-config ml simple-gp

# Train + predict
cat > gp.yaml <<'YAML'
train_path: train.csv      # CSV with columns f1,f2,...,y
target_column: y
feature_columns: [f1, f2, f3]   # optional; defaults to all non-target columns
smiles_column: smiles           # optional; auto-generates fingerprints
ignore_columns: [id, comment]   # optional; excluded from auto feature detection
fingerprint_types: [morgan, atompair, featmorgan]  # default set
kernel: tanimoto            # or rbf
noise: 0.05
# use_pca: true             # optional: reduce feature dimensions, requires kernel: rbf
# pca_n_components: 128     # optional: selects PCA output dims (default=min(n_samples, n_features))
test_path: test.csv         # optional; defaults to training data
save_report: true           # write JSON with train/test metrics
log_targets: false          # fit in log-space (requires positive targets)
generate_scatter_plot: true # render scatter if csv-scatter is available
scatter_filename: gp_scatter.html
drop_non_positive_targets: true  # when log_targets is true, drop non-positive rows instead of erroring
mode: train                # train (fit + predict) or predict (load serialized model)
model_path: simple_gp_model.joblib  # where to save/load the model bundle
id_column: id              # optional; carries IDs into predictions
YAML

caddforge run ml simple-gp --config gp.yaml

Outputs land under runs/<date>/<job-id>/ with results/predictions.csv, results/results.json (if enabled), and results/result_record.json. When log_targets is enabled, gp_variance is reported in model (log) space for interpretability; a linear-space variance is additionally stored as gp_variance_linear. Predictions are standardized to include y_pred, y_std (aliases for gp columns), and optional y_true/id/split.

If caddforge-datascience (csv-scatter) is installed and the target column is present in the predictions, the workflow also renders an optional plot (gp_scatter.html) showing target_column vs gp_mean (color-coded by observation std in linear space when log_targets is true, otherwise observation std in model space), with SMILES depictions if a smiles_column is provided).

Notes on log targets and variances

• log_targets: true fits the GP on log(target); non-positive targets are dropped by default (drop_non_positive_targets).
• gp_variance is always the model-space variance (log-space when log_targets is true). A linear-space variance derived from log-normal moments is emitted as gp_variance_linear, along with standard deviations for both latent and observation scales.
• Scatter plots switch both axes to log scale when log_targets is enabled; color uses observation std.
• When use_pca: true the feature matrix is centered/scaled (configurable) and reduced via PCA before fitting; RBF kernel is required and enforced.
• mode: predict loads a serialized model bundle (including PCA/standardization) from model_path and skips refitting; mode: train fits and saves the bundle.
• When a smiles_column is provided, SMILES are carried into predictions.csv.

Python API

from cadd_ml.simple_gp.workflow import SimpleGPConfig, SimpleGPWorkflow

cfg = SimpleGPConfig(train_path="train.csv", target_column="y", test_path="test.csv")
wf = SimpleGPWorkflow(cfg)
job = wf.prepare_job()
result = wf.run(job)
print(result.metrics)

PCA Tool

A lightweight PCA workflow (powered by scikit-learn) is available for exploratory analysis or as a preprocessing step for other tools.

cat > pca.yaml <<'YAML'
data_path: features.csv          # CSV with feature columns
# feature_columns: [f1, f2, f3]  # optional; defaults to all except ignore_columns
ignore_columns: [id]             # optional
n_components: 5                  # optional; defaults to min(n_samples, n_features)
standardize: true                # center + scale to unit variance
drop_missing_rows: true          # drop rows with missing feature values
scores_filename: pca_scores.csv
components_filename: pca_components.csv
results_filename: results.json
YAML

caddforge run ml pca --config pca.yaml

Outputs (under results/):

• pca_scores.csv: per-row principal component scores (PC1, PC2, ...).
• pca_components.csv: component loadings.
• results.json: metadata + explained variance ratios.

Python API

from cadd_ml.pca.workflow import PCAConfig, PCAWorkflow

cfg = PCAConfig(data_path="features.csv", n_components=5)
wf = PCAWorkflow(cfg)
job = wf.prepare_job()
result = wf.run(job)
print(result.metrics)

Model Zoo Tool (beta)

A general-purpose scikit-learn model zoo supporting regression/classification with a unified output format.

cat > model_zoo.yaml <<'YAML'
train_path: train.csv
target_column: y
feature_columns: [f1, f2]
model_name: random_forest   # ridge, lasso, elasticnet, bayesian_ridge, logistic_regression, random_forest, extra_trees, svm, knn, gaussian_process, xgboost
task: regression            # or classification
test_size: 0.2              # used if no test_path provided
random_state: 123
classification_thresholds: null  # optional: bin float targets into classes for classification
time_column: null                # optional: chronological split when no test_path
standardize_targets: false       # regression only: standardize y then invert outputs
target_transform: none           # regression only: none, log, log1p (applied before standardize_targets)
log_targets: false               # deprecated; use target_transform: log
drop_non_positive_targets: true  # drop invalid targets for log/log1p when enabled
positive_class: 1                # optional for classification; defaults to 1 or max class
predict_chunk_rows: null         # optional chunk size for predict mode to reduce memory
predictions_filename: predictions.csv
results_filename: results.json
metrics_filename: metrics.json
mode: train                # train or predict (predict uses model_path only)
model_path: artifacts/model.joblib
YAML

caddforge run ml model-zoo --config model_zoo.yaml

Outputs (under results/):

• predictions.csv: includes id, split, y_pred, optional y_true, smiles when present, and proba_* or y_std when available. For thresholded classification, y_true_class is added.
• metrics.json: task-specific metrics.
• results.json: summary metadata.
• Model artifact stored under artifacts/model.joblib (includes positive_class and any saved decision threshold from the explorer).
• Large predict-only runs can stream in chunks via predict_chunk_rows to reduce memory; progress is logged per chunk.

Supported models and tasks:

• Regression only: ridge, lasso, elasticnet, bayesian_ridge, gaussian_process
• Classification only: logistic_regression
• Both regression and classification: random_forest, extra_trees, svm, knn, xgboost

Python API

from cadd_ml.model_zoo.workflow import ModelZooConfig, ModelZooWorkflow

cfg = ModelZooConfig(train_path="train.csv", target_column="y", model_name="random_forest")
wf = ModelZooWorkflow(cfg)
job = wf.prepare_job()
result = wf.run(job)
print(result.metrics)

Model Factory Tool (beta)

Runs multiple model-zoo models against the same training/evaluation data and aggregates their metrics, uncertainty diagnostics, confusion summaries, and serialized model artifacts. The factory is a training workflow; predict-only operation remains a model-zoo feature.

cat > model_factory.yaml <<'YAML'
train_path: train.csv
target_column: y
task: regression
feature_columns: [f1, f2]
ignore_columns: [compound_id, measured_at]
test_path: null
smiles_column: smiles
fingerprint_types: [morgan, atompair, featmorgan]
fp_bits: 2048
fp_radius: 2
fp_use_chirality: true
model_names: [ridge, random_forest, xgboost]
model_params:
  random_forest:
    n_estimators: 500
    random_state: 123
  xgboost:
    max_depth: 6
calibrate: false
calibrate_method: sigmoid
test_size: 0.2
random_state: 123
predictions_filename: predictions.csv
results_filename: results.json
metrics_filename: metrics.json
id_column: compound_id
classification_thresholds: null
time_column: measured_at
standardize_targets: false
positive_class: null
target_transform: none
drop_non_positive_targets: true
# log_targets: false  # deprecated alias for target_transform: log
YAML

caddforge run ml model-factory --config model_factory.yaml

Set task explicitly when possible. With an explicit task and no model_names, the factory selects all registry models compatible with that task. If both are omitted, it attempts every registered model and lets each subrun infer the task, which can include incompatible models.

Configuration options

The following are all model-factory-specific options. Common CADDForge settings such as backend, run_directory, registry_path, job_name, result export, and Slurm settings are inherited from BaseConfig.

Option	Type and default	Description
train_path	path, required	Training CSV. It must contain target_column and all configured feature, SMILES, ID, and time columns used for training.
target_column	string, required	Column to predict. Numeric values are required for regression and for threshold-binned classification.
task	regression, classification, or null; default null	Task passed to every model-zoo subrun. When omitted, each subrun infers classification for thresholded or non-numeric targets and regression otherwise. Explicitly setting it also lets the factory choose only compatible default models.
feature_columns	list of strings or null; default null	Numeric tabular features. When omitted, all training columns except the target, SMILES column, and ignore_columns are used. These features are concatenated with fingerprints when smiles_column is also set.
ignore_columns	list of strings; default []	Columns excluded only from automatic feature discovery. It has no effect when feature_columns is explicit.
test_path	path or null; default null	Optional external evaluation CSV. When set, all training rows are fitted and this file is predicted. Otherwise a holdout is taken from train_path.
smiles_column	string or null; default null	SMILES column used to generate RDKit fingerprints. Fingerprints may be used alone or appended to numeric feature_columns.
fingerprint_types	list of strings; default [morgan, atompair, featmorgan]	Fingerprints to concatenate for each SMILES. Supported values are morgan, atompair, and featmorgan. Used only with smiles_column.
fp_bits	integer; default 2048	Bit count generated for each fingerprint type. The total fingerprint width is this value times the number of fingerprint types.
fp_radius	integer; default 2	Radius for Morgan and feature-Morgan fingerprints.
fp_use_chirality	boolean; default true	Include chirality in generated fingerprints.
model_names	list of strings or null; default null	Models to run. Accepted names are ridge, lasso, elasticnet, bayesian_ridge, logistic_regression, random_forest, extra_trees, svm, knn, gaussian_process, and xgboost. With an explicit task, null selects all compatible models.
model_params	mapping; default {}	Per-model estimator keyword arguments keyed by model name, for example random_forest: {n_estimators: 500}. Values are passed directly to the underlying scikit-learn or XGBoost constructor.
calibrate	boolean; default false	For classification, wrap each estimator in three-fold CalibratedClassifierCV before prediction. Ignored for regression.
calibrate_method	sigmoid or isotonic; default sigmoid	Probability calibration method used when calibrate is true.
test_size	float from 0.0 to 0.9; default 0.2	Fraction reserved for evaluation when test_path is absent; use a positive value when a holdout is required. A chronological holdout is used when time_column is set; otherwise the split is random and classification is stratified when possible.
random_state	integer; default 123	Seed for the random holdout split. It is not automatically injected into estimators; set estimator seeds in model_params when needed.
predictions_filename	string; default predictions.csv	Predictions filename inside each model subrun's results/ directory.
results_filename	string; default results.json	Model-zoo summary filename inside each model subrun's results/ directory.
metrics_filename	string; default metrics.json	Metrics filename inside each model subrun's results/ directory.
id_column	string or null; default null	Source column copied into per-model predictions as id. Row indices are used when omitted.
classification_thresholds	list of floats or null; default null	Sorted cut points used with classification to bin a continuous target via numpy.digitize; N thresholds create N+1 integer classes. A factory confusion summary is emitted only for binary outcomes.
time_column	string or null; default null	Datetime-like column used for a chronological split when test_path is absent. The newest test_size fraction becomes the holdout.
standardize_targets	boolean; default false	Regression only. Standardize the target after any transform, fit in standardized space, then invert predictions and reported metrics to the original scale.
positive_class	integer or null; default null	Binary classification label treated as positive for probability and confusion diagnostics. Model-zoo subruns prefer class 1, then the highest class, when omitted; the factory confusion summary defaults to 1.
log_targets	boolean; default false	Deprecated compatibility alias for target_transform: log. Do not set it together with a non-none target_transform.
target_transform	none, log, or log1p; default none	Regression only. Transform targets before optional standardization and invert predictions back to the original scale. log requires values greater than zero; log1p requires values greater than -1.
drop_non_positive_targets	boolean; default true	Drop invalid training targets before log or log1p transformation. When false, invalid values fail the run.

Model compatibility by task:

Task	Models
Regression only	ridge, lasso, elasticnet, bayesian_ridge, gaussian_process
Classification only	logistic_regression
Both	random_forest, extra_trees, svm, knn, xgboost

Outputs in the factory run directory:

• summary.csv and summary.json: one row per model, with metric_* columns, optional uncertainty_correlation, and binary confusion_* values when available. Regression summaries also add OLS slope, intercept, R2, and RMSE diagnostics.
• <model-name>/: the complete model-zoo subrun, including its configured predictions, metrics, results, logs, and artifact.
• artifacts/<model-name>_model.joblib: pooled copies of successfully produced model bundles.
• results/result_record.json: canonical factory result. Its paths expose summary_csv, summary_json, and log.

Python API

from cadd_ml.model_factory.workflow import ModelFactoryConfig, ModelFactoryWorkflow

cfg = ModelFactoryConfig(train_path="train.csv", target_column="y", task="regression")
wf = ModelFactoryWorkflow(cfg)
job = wf.prepare_job()
result = wf.run(job)
print(result.metrics)

Explore Classifier Tool (alpha)

Interactive threshold tweaking for classification runs (model-zoo or model-factory).

cat > explore_classifier.yaml <<'YAML'
job_id: <existing_job_id>
run_base: runs
# model_name: random_forest   # optional; required for model-factory to preselect
positive_class: 1             # optional; defaults to 1
YAML

caddforge run ml explore-classifier --config explore_classifier.yaml

• Opens a Streamlit UI to select classifiers (for factory runs), adjust decision threshold (binary only), view confusion matrix and metrics, and save the chosen threshold.
• Saves threshold JSON under artifacts/ and also updates the model artifact with the decision threshold when possible (used automatically by model-zoo/model-factory in predict mode).
• Provides metric vs threshold plots (0.01 steps) to visualize how metrics move with the decision threshold.
• Requires streamlit and plotly installed locally (install with the ui extra or add manually).

Python API

from cadd_ml.explore_classifier.workflow import ExploreClassifierConfig, ExploreClassifierWorkflow

cfg = ExploreClassifierConfig(job_id="existing_job_id", run_base="runs")
wf = ExploreClassifierWorkflow(cfg)
job = wf.prepare_job()
result = wf.run(job)  # launches Streamlit UI

Next Steps

• Extend configs for model persistence and cross-validation.
• Add richer kernels and hyperparameter tuning helpers.
• Document feature preprocessing expectations for different kernels.

Compressed inputs

Training, test, and PCA CSV inputs may be supplied as .csv.gz. They are expanded in job-local runtime scratch, and saved configurations continue to reference the compressed source.

Continuous integration and releases

.github/workflows/release.yml builds a wheel and source archive, validates metadata, and tests the installed wheel on Linux with Python 3.10 and 3.12. The lightweight test selection is recorded in scripts/ci_tests.json; external scientific engines are validated separately.

Pushing a matching v* release tag publishes the tested artifacts to PyPI. Branch pushes, pull requests and manual runs never publish. Configure a PyPI Trusted Publisher for the idorsia repository, release.yml workflow and pypi environment before pushing release tags. Runtime CADDForge dependencies must already be available publicly; publish core first, and publish docking/descriptors before MD.