optimized-rounder 0.1.2

An optimizer for finding optimal thresholds in ordinal classification problems

Optimized Rounder

An efficient optimizer for finding optimal thresholds in ordinal classification problems. This package uses Optuna for efficient threshold search with support for cross-validation and multiple evaluation metrics.

Installation

pip install optimized-rounder

Features

• Threshold Optimization: Find optimal thresholds to convert continuous predictions to discrete classes
• Multiple Metrics: Support for various evaluation metrics including quadratic kappa, linear kappa, RMSE, accuracy, and F1 scores
• Cross-Validation: Built-in support for K-fold and stratified cross-validation
• Efficient Search: Uses Optuna for efficient Bayesian optimization of thresholds
• Comprehensive Evaluation: Evaluate models using multiple metrics simultaneously

Quick Start

from oprounder import OptimizedRounder
import numpy as np
from sklearn.metrics import cohen_kappa_score

# Generate synthetic data
np.random.seed(42)
n_classes = 4
n_samples = 1000
y_true = np.random.randint(0, n_classes, size=n_samples)
output = y_true + np.random.normal(0, 0.9, size=n_samples) # dummy model output

# Initialize and fit the optimizer
rounder = OptimizedRounder(n_classes=n_classes, n_trials=100)
rounder.fit(output, y_true)

# Get the optimal thresholds
print(f'Optimal thresholds: {rounder.thresholds}')

# Make predictions
y_pred = rounder.predict(output)
kappa = cohen_kappa_score(y_true, y_pred, weights='quadratic')
print(f'Optimal Quadratic kappa: {kappa:.4f}')
y_pred_default = rounder.apply_thresholds(output, rounder.default_thresholds) # [0.5, 1.5, 2.5, 3.5]
kappa_default = cohen_kappa_score(y_true, y_pred_default, weights='quadratic')
print(f'Default Quadratic kappa: {kappa_default:.4f}')

Advanced Usage

With Cross-Validation

# Use 5-fold stratified cross-validation
rounder = OptimizedRounder(
    n_classes=4,
    n_trials=200,
    cv=5,
    stratified=True,
    metric='quadratic_kappa',
    verbose=True
)

rounder.fit(output, y_true)
print(f'CV Results: {rounder.cv_results_}')

Using Different Metrics

# Optimize for F1 weighted score
rounder = OptimizedRounder(
    n_classes=4,
    n_trials=200,
    metric='f1_weighted'
)

rounder.fit(output, y_true)

# Comprehensive evaluation
output_val = y_true + np.random.normal(0, 0.8, size=n_samples)
metrics = rounder.evaluate(output_val, y_true)
for metric_name, value in metrics.items():
    print(f"{metric_name}: {value:.4f}")

API Reference

OptimizedRounder

OptimizedRounder(
    n_classes=None,
    n_trials=200,
    cv=None,
    stratified=True,
    metric='quadratic_kappa',
    verbose=False,
    random_state=42
)

Parameters

• n_classes: Number of target classes (0, 1, 2, ..., n_classes-1)
• n_trials: Number of optimization trials for Optuna
• cv: Number of cross-validation folds or a CV splitter object
• stratified: Whether to use stratified CV (only when cv is an integer)
• metric: Metric to optimize ('quadratic_kappa', 'linear_kappa', 'rmse', 'accuracy', 'f1_macro', 'f1_weighted', 'f1_micro')
• verbose: Whether to display Optuna's optimization progress
• random_state: Random seed for reproducibility

Methods

• fit(X, y): Find optimal thresholds using Optuna optimization
• predict(X): Convert continuous predictions to discrete classes using optimal thresholds
• fit_predict(X, y): Train the optimizer and return predictions in one step
• coefficients(): Get the optimal thresholds found during training
• evaluate(X, y): Evaluate the model on multiple metrics

Use Cases

• Regression to Classification: Convert regression outputs to discrete classes
• Ordinal Classification: Optimize thresholds for ordinal targets
• Ensemble Calibration: Calibrate probability outputs from ensemble models
• Competition Metrics: Optimize directly for competition metrics like quadratic kappa

License

This project is licensed under the MIT License - see the LICENSE file for details.