SVMargin 1.3

A package for cost-sensitive multiclass classification that increases the sensitivity of important classes by shifting the decision boundary between them according to a prioritization vector.

SVMargin

Version: 0.1.0
License: MIT License
Author: Eran Kaufman
Email: [e-mail]

Overview

SVMargin is a Python package designed to address the problem of imbalanced data and cost-sensitive multiclass classification. The package increases the sensitivity of important classes by shifting the decision boundary between them according to a prioritization vector. This results in a tighter error bound for critical classes while reducing overall out-of-sample error.

The package supports various kernel methods, including linear, RBF, and polynomial kernels, and is adaptable to neural networks. It also includes generalization bounds and demonstrates Fisher consistency.

Features

• Imbalanced Classification: Adjusts the imbalance of classes based on their inverse size.
• Cost-Sensitive Classification: Adjusts the sensitivity of classes based on their importance.
• Apportioned Margin Framework: Efficiently shifts decision boundaries according to a prioritization vector.
• Support for Multiple Kernels: Includes linear, RBF, and polynomial kernels.
• Neural Network Adaptation: Can be integrated with neural networks.
• Fisher Consistency: Ensures the consistency of the classifier.
• Generalization Bounds: Provides theoretical guarantees on performance.

Installation

You can install the package directly from PyPI using pip:

pip install SVMargin

Usage

Here is a basic example of how to use the package:

linear comparison for different costs

import numpy as np
from SVMargin import ImbalancedSVM
from sklearn.preprocessing import MinMaxScaler
import matplotlib.pyplot as plt

# Generate synthetic data
N = 20
s = np.random.normal(0, 0.5, (N, 2))
X1 = s + [0, 0]
X2 = s + [0, 6]
X3 = s + [6, 6]
X4 = s + [6, 0]

X = np.concatenate([X1, X2, X3, X4])
y = np.concatenate([0 * np.ones(N), 1 * np.ones(N), 2 * np.ones(N), 3 * np.ones(N)])
y = y.astype(int)

# Define class weights
thetas = np.array([
    [1, 1, 1, 1],
    [10, 1, 1, 1],
    [10, 10, 1, 1],
    [10, 10, 10, 1],
])

# Normalize the data
X = MinMaxScaler().fit_transform(X)

# Train and plot the decision boundaries for each set of weights
fig, sub = plt.subplots(2, 2)
for theta, title, ax in zip(thetas, thetas, sub.flatten()):
    cls = ImbalancedSVM(kernel='linear', class_weight=theta, verbose= True)
    cls.fit(X, y)
    plot_contours(ax, cls, xx, yy, xy, cmap=plt.cm.coolwarm, alpha=0.8)
    ax.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.coolwarm, s=20, edgecolors='k')
    ax.set_xlim(xx.min(), xx.max())
    ax.set_ylim(yy.min(), yy.max())
    ax.set_xticks(())
    ax.set_yticks(())
    ax.set_title(title)
plt.show()

polynomial kernel example

#polynomial kernel example
N=100
X = np.random.normal(0, 0.5,(N,2))
y=np.zeros(N)
for index in range(N):
    if ((X[index][0]**2+X[index][1]**2)<0.3):
        y[index]=1
    else:
        y[index]=-1
xx, yy = make_meshgrid(X[:, 0], X[:, 1])
xy=np.c_[xx.ravel(), yy.ravel()]
fig, sub = plt.subplots(1,2)
for theta, title, ax in zip(thetas,thetas, sub.flatten()):
    cls = ImbalancedSVM(kernel='poly',class_weight=theta, verbose= True)
    cls.fit(X,y)
    plot_contours(ax, cls, xx, yy,xy,cmap=plt.cm.coolwarm, alpha=0.8)
    ax.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.coolwarm, s=20, edgecolors='k')

  #  ax.set_xlim(xx.min(), xx.max())
  #  ax.set_ylim(yy.min(), yy.max())

    ax.set_xticks(())
    ax.set_yticks(())
    ax.set_title(title)
    

plt.show()

Available Kernels

• Linear Kernel: linear
• RBF Kernel: rbf
• Polynomial Kernel: poly

You can select the kernel by specifying the kernel parameter when creating an instance of ImbalancedSVM.

Class Weights

The class_weight parameter allows you to assign different importance to different classes. It accepts a dictionary where keys are class labels, and values are the corresponding weights.

Examples

Here are a few usage examples:

Linear Kernel Example

cls = ImbalancedSVM(kernel='linear', class_weight={0: 10, 1: 1}, verbose= True)
cls.fit(X, y)
predictions = cls.predict(X_test)

RBF Kernel Example

cls = ImbalancedSVM(kernel='rbf', gamma=0.5, class_weight={0: 1, 1: 10}, verbose= True)
cls.fit(X, y)
predictions = cls.predict(X_test)

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgements

• scikit-learn for providing the foundational tools for machine learning in Python.
• Matplotlib for data visualization.

References

If you would like to learn more about the theoretical background and the research behind this package, please refer to the following article: Please cite this article when using this package!

• Apportioned Margin Approach for Cost Sensitive Large Margin Classifiers
  Gottlieb, LA., Kaufman, E. & Kontorovich, A. Apportioned margin approach for cost sensitive large margin classifiers. Ann Math Artif Intell 89, 1215–1235 (2021). https://doi.org/10.1007/s10472-021-09776-w

Contact

If you have any questions or suggestions, feel free to reach out at [e-mail].