MultiModel Analysis

MultiModel Analysis is a Python framework designed to automate model benchmarking, evaluation, and visualization for Supervised Machine Learning tasks (Classification and Regression). It provides a unified interface for training multiple baseline models, computing statistical evaluation metrics, generating diagnostic visualizations, and identifying optimal model candidates.

Table of Contents

• Overview
• Design Architecture
• Supported Estimators
• Installation
• Quick Start Guide
  • Classification Pipeline
  • Regression Pipeline
• API Reference & Complete Function List
  • Standalone Utility Functions
  • MultiModelClassifier
  • MultiModelRegressor
• Evaluation Metrics
• Dependencies
• License and Citation

Overview

Selecting an appropriate estimator for a given tabular dataset requires benchmarking multiple baseline algorithms. Manually executing model fitting, cross-validation, feature scaling, metric aggregation, and plotting leads to repetitive code overhead.

multimodel_analysis streamlines this process by implementing an automated execution pipeline:

1. Automated Preprocessing: Applies standard scaling (StandardScaler) while preserving pandas DataFrame structure and column metadata.
2. Label Encoding: Encodes target arrays (LabelEncoder) to support numerical, string, and categorical target types across binary and multiclass tasks.
3. Stratified Splitting: Implements stratified train-test splits (train_test_split) for classification tasks to maintain target class proportions.
4. Fault-Tolerant Training: Wraps individual model evaluations in isolated execution blocks to prevent full execution failure if a single estimator encounters a fitting exception.
5. Metric Calculation & Visualization: Computes standardized evaluation metrics and renders diagnostic plots (Confusion Matrices, ROC Curves, Residual/Prediction Scatter plots, and Bar Charts).
6. Headless & File Exporting: Supports saving report tables directly to CSV, Excel, HTML, or JSON, and saving plot charts to PNG files without requiring an interactive display window.

Design Architecture

flowchart TD
    InputData["Input Dataset (X, y)"] --> DataPrep["Target & Feature Preprocessing"]
    
    DataPrep --> Encoder["LabelEncoder (Target Encoding)"]
    Encoder --> ScalingCheck{"Feature Scaling Enabled?"}
    
    ScalingCheck -- Yes --> Scaler["StandardScaler (Preserves DataFrame Metadata)"]
    ScalingCheck -- No --> RawFeatures["Unscaled Features"]
    
    Scaler --> Split["train_test_split (Stratified for Classification)"]
    RawFeatures --> Split
    
    Split --> ModelExecution["Parallel Model Execution & Custom Models"]
    
    ModelExecution --> MetricsEngine["Metrics Engine & Diagnostic Evaluation"]
    
    MetricsEngine --> TabularReport["Tabular Report & Model Recommendation"]
    MetricsEngine --> Visualizations["Visualizations (CM, ROC, Scatter, Bar Charts)"]
    MetricsEngine --> FileExport["File Export (CSV, Excel, HTML, PNG plots)"]

Supported Estimators

Classification Estimators

• Logistic Regression (LogisticRegression)
• Support Vector Machine (SVC)
• Decision Tree Classifier (DecisionTreeClassifier)
• K-Nearest Neighbors (KNeighborsClassifier)
• Gaussian Naive Bayes (GaussianNB)
• Random Forest Classifier (RandomForestClassifier)
• Gradient Boosting Classifier (GradientBoostingClassifier)
• AdaBoost Classifier (AdaBoostClassifier)
• Custom Estimators: User-supplied scikit-learn compatible classifiers.

Regression Estimators

• Linear Regression (LinearRegression)
• Lasso Regression (Lasso)
• Ridge Regression (Ridge)
• Support Vector Regressor (SVR)
• Decision Tree Regressor (DecisionTreeRegressor)
• Random Forest Regressor (RandomForestRegressor)
• Gradient Boosting Regressor (GradientBoostingRegressor)
• Custom Estimators: User-supplied scikit-learn compatible regressors.

Installation

Installation via PyPI (Recommended)

pip install multimodel-analysis

Installation from Source

pip install git+https://github.com/udityamerit/Multimodel-Analysis-Pacakge.git

Quick Start Guide

Classification Pipeline

import pandas as pd
from multimodel_analysis import MultiModelClassifier, save_report

# 1. Prepare features and target variable
df = pd.read_csv("dataset.csv")
X = df.drop(columns=["target"])
y = df["target"]

# 2. Instantiate MultiModelClassifier
classifier = MultiModelClassifier(
    X=X,
    y=y,
    test_size=0.3,
    scaled_data=True,
    random_state=42,
    stratify=True,
    n_jobs=-1
)

# 3. Train all classification models (including optional custom models)
results = classifier.run_all_models()

# 4. Display tabular performance report
df_report = classifier.show_tabular_report(results, return_df=True)

# 5. Export tabular report to disk
save_report(df_report, "classification_report.csv")

# 6. Render & save diagnostic figures
classifier.plot_confusion_matrices(results, save_path="confusion_matrix.png")
classifier.plot_roc_curves(results, save_path="roc_curve.png")
classifier.plot_comparison(results, save_path="metrics_comparison.png")

# Or run everything in one line:
classifier.get_summary(results, save_prefix="clf_run")

Regression Pipeline

import pandas as pd
from multimodel_analysis import MultiModelRegressor

# 1. Prepare features and target variable
df = pd.read_csv("housing.csv")
X = df.drop(columns=["Price"])
y = df["Price"]

# 2. Instantiate MultiModelRegressor
regressor = MultiModelRegressor(
    X=X,
    y=y,
    test_size=0.3,
    scaled_data=True,
    random_state=42,
    n_jobs=-1
)

# 3. Train all regression models
results = regressor.run_all_models()

# 4. Display tabular performance report
regressor.show_tabular_report(results)

# 5. Render & save diagnostic figures
regressor.plot_true_vs_predicted(results, save_path="true_vs_pred.png")
regressor.plot_comparison(results, save_path="r2_comparison.png")

# Or run everything and export outputs in one line:
regressor.get_summary(results, save_prefix="reg_run")

API Reference & Complete Function List

Standalone Utility Functions

save_report(df, filepath)

Saves a model comparison DataFrame to disk. Automatically infers format from file extension.

from multimodel_analysis import save_report

save_report(df, "report.csv")   # Saves as CSV
save_report(df, "report.xlsx")  # Saves as Excel Spreadsheet
save_report(df, "report.html")  # Saves as HTML Table
save_report(df, "report.json")  # Saves as JSON File

• df (pandas.DataFrame): Report DataFrame returned by show_tabular_report(models, return_df=True).
• filepath (str): Target file path with .csv, .xlsx, .xls, .html, .htm, or .json extension.

MultiModelClassifier

multimodel_analysis.MultiModelClassifier(
    X, 
    y, 
    test_size=0.3, 
    scaled_data=False, 
    random_state=42, 
    stratify=True,
    n_jobs=-1
)

Constructor Parameters

• X (pandas.DataFrame or numpy.ndarray): Feature matrix of shape (n_samples, n_features).
• y (pandas.Series, pandas.DataFrame, or numpy.ndarray): Target labels (binary or multiclass, numerical or string).
• test_size (float, default=0.3): Proportion of dataset for test split (between 0.0 and 1.0).
• scaled_data (bool, default=False): Fits and applies StandardScaler to features while preserving DataFrame columns and index metadata.
• random_state (int, default=42): Seed for train-test split and reproducible model initialization.
• stratify (bool, default=True): Performs stratified splitting when class sample counts allow (min_count >= 2).
• n_jobs (int or None, default=-1): Number of parallel CPU threads for estimators supporting n_jobs.

Complete Method List

Individual Model Methods

• clf.Logistic_model() : Trains and evaluates Logistic Regression.
• clf.Support_vector_model() : Trains and evaluates Support Vector Classifier.
• clf.DecisionTree_model() : Trains and evaluates Decision Tree Classifier.
• clf.KNN_model() : Trains and evaluates K-Nearest Neighbors.
• clf.Naive_Bayes_model() : Trains and evaluates Gaussian Naive Bayes.
• clf.RandomForest_model() : Trains and evaluates Random Forest Classifier.
• clf.GradientBoosting_model() : Trains and evaluates Gradient Boosting Classifier.
• clf.AdaBoost_model() : Trains and evaluates AdaBoost Classifier.

MultiModelRegressor

multimodel_analysis.MultiModelRegressor(
    X, 
    y, 
    test_size=0.3, 
    scaled_data=False, 
    random_state=42,
    n_jobs=-1
)

Constructor Parameters

• X (pandas.DataFrame or numpy.ndarray): Feature matrix of shape (n_samples, n_features).
• y (pandas.Series, pandas.DataFrame, or numpy.ndarray): Continuous target values (automatically flattened if 2D single-column input).
• test_size (float, default=0.3): Proportion of dataset for test split.
• scaled_data (bool, default=False): Applies StandardScaler to features retaining DataFrame structure.
• random_state (int, default=42): Seed for reproducible train-test split.
• n_jobs (int or None, default=-1): Number of CPU threads for parallel regressors.

Complete Method List

Individual Model Methods

• reg.LinearRegression_model() : Trains and evaluates Linear Regression.
• reg.Lasso_model() : Trains and evaluates Lasso Regression.
• reg.Ridge_model() : Trains and evaluates Ridge Regression.
• reg.SVR_model() : Trains and evaluates Support Vector Regressor.
• reg.DecisionTree_model() : Trains and evaluates Decision Tree Regressor.
• reg.RandomForest_model() : Trains and evaluates Random Forest Regressor.
• reg.GradientBoosting_model() : Trains and evaluates Gradient Boosting Regressor.

Class Aliases

• MultiModelRegressior: Backward-compatibility alias for MultiModelRegressor.

Evaluation Metrics

Classification Metrics

• Accuracy: $\frac{TP + TN}{TP + TN + FP + FN}$
• Precision (Weighted): $\sum_{c} w_c \cdot \frac{TP_c}{TP_c + FP_c}$
• Recall (Weighted): $\sum_{c} w_c \cdot \frac{TP_c}{TP_c + FN_c}$
• F1 Score (Weighted): $2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$
• ROC-AUC: Computed using positive-class probabilities for binary tasks and One-vs-Rest (ovr) weighted macro-strategy for multiclass tasks.

Regression Metrics

• Mean Absolute Error (MAE): $\frac{1}{n} \sum_{i=1}^n |y_i - \hat{y}_i|$
• Mean Squared Error (MSE): $\frac{1}{n} \sum_{i=1}^n (y_i - \hat{y}_i)^2$
• Root Mean Squared Error (RMSE): $\sqrt{\frac{1}{n} \sum_{i=1}^n (y_i - \hat{y}_i)^2}$
• Coefficient of Determination ($R^2$): $1 - \frac{\sum_{i=1}^n (y_i - \hat{y}i)^2}{\sum{i=1}^n (y_i - \bar{y})^2}$

Dependencies

• python >= 3.8
• numpy
• pandas
• matplotlib
• seaborn
• scikit-learn

License and Citation

This project is licensed under the Apache Software License 2.0.

Author: Uditya Narayan Tiwari
Repository: https://github.com/udityamerit/Multimodel-Analysis-Pacakge
PyPI Package: https://pypi.org/project/multimodel-analysis/