ogboost 0.5.0

Gradient Boosting Ordinal Regression

Gradient Boosting for Ordinal Regression (GBOR)

gbor is a scikit-learn-compatible, Python package designed for gradient boosting tailored to ordinal regression problems. It does so by alternating between fitting a base learner - GradientBoostingRegressor, by default - to predict a latent score that specifies the mean of a cumulative density function for the observed discrete outcomes, and fitting a set of thresholds that convert the latent score to the discrete outcome. It can be viewed as coordinate-descent optimization that combines functional gradient descent (the boosting stage) with ordinary gradient descent (for adjusting thresholds).

The main class in the package, GradientBoostingOrdinal, supports custom link functions, sample weighting, early stopping criteria, and staged predictions. In addition to methods for predicting class labels and probabilities, similar to nominal classifiers, the decision_function method of the class predicts the latent score, which can be used to achieve superior performance in discreiminative/ranking tasks.

Features

• Fully compatible with scikit-learn pipelines.
• Customizable link functions: Probit, Logit, and Complementary Log-Log.
• Supports sample weighting for robust handling of imbalanced datasets.
• Subsampling for stochastic gradient boosting.
• Early stopping based on validation loss.
• Utilities for data loading and evaluation metrics.

Installation

Clone the repository and install the package:

git clone https://github.com/yourusername/gbor.git
cd gbor
pip install .

Quick Start

Load the Wine Quality Dataset

The package includes a utility to load the wine quality dataset from the UCI repository.

from gbor.data import load_wine_quality

# Load data
red_wine, white_wine = load_wine_quality()
X = red_wine.drop(columns="quality")
y = red_wine["quality"]

Train a Gradient Boosting Ordinal Model

from gbor.main import GradientBoostingOrdinal

# Initialize and fit the model
model = GradientBoostingOrdinal(n_estimators=100, link_function='logit', verbose=1)
model.fit(X, y)

# Predict class labels and probabilities
predicted_labels = model.predict(X)
predicted_probabilities = model.predict_proba(X)

# Evaluate using the concordance index
from gbor.main import concordance_index
c_index = concordance_index(y, model.decision_function(X))
print(f"Concordance Index: {c_index:.3f}")

License

This package is licensed under the MIT License.

---

For issues or contributions, feel free to create a pull request or open an issue on the GitHub repository.