Radiomics Modelling Suite

Radiomics Modelling Suite (RMS) is for building and evaluating radiomics machine learning models from raw medical images and corresponding segmentations using Python. Each stage of the RMS outputs a PDF report reporting the details of that stage.

RMS is designed to aid adherence to the RQS and METRICS criteria, developed by Lambin et al. and Kocak et al. respectively, by automatically reporting the information required for a high RQS or METRICS score. This will increase transparency in reporting of radiomics studies, leading to increased reliability of the results of the study.

RMS is capable of processing PET, MRI, or CT medical images in dicom format. Models can be trained from radiomics features extracted from different imaging modalities, such as radiomics features from PET and MRI data from the same patient. Binary classification and survival model training is supported.

Not approved for clinical use.

Stages of the Radiomics Pipeline

RMS incorporates the following stages of the radiomics pipeline.

1. Sample size calculation
2. Data preparation
3. Image pre-processing
4. Feature extraction
5. Feature Selection
6. Model Training
7. Model Evaluation

Sample Size Calculation

There are two modules present for sample size calculation: one which calculates the sample size for a prospective study, and another which calculates the sample size for a validation study.

Data Preparation

RMS reads DICOM medical images and registers them to their corresponding NIfTI segmentations before converting them to NIfTI files. The acquisition parameters from the DICOM files are reported as a PDF and are used to calculate the SUV values in PET images. If multiple segmentations for a given patient are specified (for example, automatic and manual segmentations), the segmentations can be compared using a variety of metrics.

Image Pre-Processing

The available pre-processing modules include image normalisation, gray-level discretisation, and resampling. Two methods of normalisation are available: normalisation of values to a fixed range specified by the user and normalisation of values to their Z-score. For gray-level discretisation, the user can specify the bin width or the number of bins for the gray-levels they desire.

Feature Extraction

The feature extraction module was built using the feature extractor implemented by PyRadiomics, which automates the extraction of features which adhere to the IBSI standard for radiomics feature extraction. In addition to the features extracted by PyRadiomics, the software also extracts the normalised hotspot to centroid distance, which was found to be predictive of survival in Chen et al.'s study (2025).

Feature Selection

Feature selection is crucial for radiomics models as radiomics data is high-dimensional (number of features is greater than sample size), leading to trained radiomics models overfitting to the training data. Harmonisation using the ComBat and HarMSTD methods is supported to account for inter-scanner variability. Features can be filtered by their robustness (comparing feature values between different segmentations of the same patient), and by their redundancy (only keeping predictive features which are not highly correlated with each other). Principal component analysis is also supported: however, this is not strictly a feature selection method as the features are transformed to linear combinations before dimensionality reduction is performed.

Model Training

The model training module supports the training of a logistic regression, decision tree, SVM, random forest, or XGBoost model for classification problems. Cross-validation to optimise hyperparameters using grid search is optional. Survival analysis using the Kaplan-Meier estimator, the Cox proportional hazards model, and accelerated failure time models is also supported using the Python lifelines package.

Model Evaluation

There are four stages of model evaluation supported by RMS:

1. Discrimination analysis

   Selects the optimal threshold for classification and evaluates model's performance.

2. Model calibration analysis

   Evaluates the agreement between the model's prediction of risk and the observed risk.

3. Decision curve analysis

   Quantifies the clinical value of the model as a diagnostic test.

4. Model explainability

   Helps the researcher understand the model's reasoning behind its predictions, improving model trustworthiness.

Quality Scores

RQS

Using all modules of RMS will result in a RQS score of at least 33.3% if automatic segmentations are not available and 36.1% if both automatic and manual segmentations are available. The RQS 2.0 score at Radiomics Readiness Level (RRL) 9 is at least 26.7% if automatic segmentations are not available and 32.1% if both automatic and manual segmentations are available.

METRICS

Using all modules of RMS will result in a METRICS score of at least 51.1% if automatic segmentations are not available and 52.2% if both automatic and manual segmentations are available.