CADDForge Data Science Tooling

Install this plugin with python -m pip install caddforge-datascience. It depends directly on caddforge-core>=0.21.0; Python imports and CLI commands still use caddforge. For the complete public toolbox, including user-facing extras, install python -m pip install caddforge. When migrating an existing environment, uninstall the former caddforge core distribution before installing renamed core and updated plugins.

caddforge-datascience hosts lightweight data science utilities as CADDForge plugins. The first tool, CSV Scatter, builds an interactive Bokeh scatter plot from a CSV/TSV file with optional color mapping, molecule hover images from SMILES, and threshold sliders that drive a live confusion matrix. Axis scales can be linear/log independently, and the color map can be inverted.

The confusion matrix treats the x-threshold as the predictor and the y-threshold as the actual outcome; you can control which side of each threshold is considered positive via x_positive_direction and y_positive_direction.

Tool: CSV Scatter

• Section/Name: datascience / csv-scatter
• Purpose: Generate an interactive scatter plot from a CSV/TSV file with optional color mapping, hover details (including rendered SMILES depictions), and threshold sliders (confusion matrix).
• Outputs: HTML plot under the job run directory plus a standard result_record.json.

Config example (YAML)

csv_path: data/metrics.csv
x_column: dG_tr_calc
y_column: pappab
color_column: efflux_ratio          # optional numeric column
smiles_column: smiles               # optional for hover images (requires RDKit)
id_column: idor_no                  # optional identifier shown on hover
hover_columns: [compute_time]       # any extra columns to show in hover
x_scale: linear                     # linear or log
y_scale: log                        # linear or log
invert_color_scale: false           # reverse the colormap if true
palette: viridis                    # viridis|plasma|inferno|magma|cividis|turbo|blues
thresholds_enabled: true            # show sliders + live confusion matrix
x_positive_direction: low           # which side of x threshold is positive: low|high
y_positive_direction: high          # which side of y threshold is positive: low|high
point_size: 10
alpha: 0.8
title: "Transport predictions"
run_directory: runs/demo_plot       # optional override

CLI usage

# Show a template config
caddforge show-config datascience csv-scatter

# Run a plot job
caddforge run datascience csv-scatter --config plot.yaml

# Inspect job artifacts
caddforge status <job-id>
caddforge logs <job-id>
ls runs/<date>/<job-id>/results

Python usage

from cadd_datascience import CsvScatterConfig, CsvScatterWorkflow

config = CsvScatterConfig(
    csv_path="data/metrics.csv",
    x_column="molecular_weight",
    y_column="pIC50",
    color_column="logS",
    x_scale="log",
)
workflow = CsvScatterWorkflow(config)
job = workflow.prepare_job()
result = workflow.run(job)
print(result.paths["plot_png"])

Tool: Distribution Diagnostics

• Section/Name: datascience / distribution-diagnostics
• Purpose: Diagnose whether a numeric column looks approximately normal or log-normal using summary stats, normality tests, and saved plots (histograms + Q–Q plots).
• Outputs: results/results.json report and results/distribution_diagnostics.png plot under the job run directory.

Config example (YAML)

csv_path: data.csv
column: value
bins: 50
clip_std: null
sample_size: 5000
random_seed: 0
include_log: true
plot_filename: distribution_diagnostics.png
results_filename: results.json
run_directory: runs/dist_demo

CLI usage

caddforge show-config datascience distribution-diagnostics
caddforge run datascience distribution-diagnostics --config dist.yaml
ls runs/<date>/<job-id>/results

Python usage

from cadd_datascience import DistributionDiagnosticsConfig, DistributionDiagnosticsWorkflow

cfg = DistributionDiagnosticsConfig(csv_path="data.csv", column="value")
wf = DistributionDiagnosticsWorkflow(cfg)
job = wf.prepare_job()
result = wf.run(job)
print(result.paths["report_json"])
print(result.paths["plot_png"])

Development

• Plugins are registered via the caddforge.tools entry point group.
• Tests live under tests/ and can be run with pytest once dependencies are installed.

Compressed inputs

csv-scatter accepts .csv.gz and .tsv.gz, while distribution-diagnostics accepts .csv.gz, through normal CLI, GUI, and pipeline submission. CADDForge expands them only in job-local runtime scratch; configuration files continue to reference the original compressed file.

Continuous integration and releases

.github/workflows/release.yml builds a wheel and source archive, validates metadata, and tests the installed wheel on Linux with Python 3.10 and 3.12. The lightweight test selection is recorded in scripts/ci_tests.json; external scientific engines are validated separately.

Pushing a matching v* release tag publishes the tested artifacts to PyPI. Branch pushes, pull requests and manual runs never publish. Configure a PyPI Trusted Publisher for the idorsia repository, release.yml workflow and pypi environment before pushing release tags. Runtime CADDForge dependencies must already be available publicly; publish core first, and publish docking/descriptors before MD.