Citing

If you use this package in your research, please cite the paper using the following BibTex entry:

@article{van2017fast,
  title={Fast Meta-Learning for Adaptive Hierarchical Classifier Design},
  author={Gerrit J.J. van den Burg and Alfred O. Hero},
  journal={arXiv preprint arXiv:1711.03512},
  archiveprefix={arXiv},
  year={2017},
  eprint={1711.03512},
  url={https://arxiv.org/abs/1711.03512},
  primaryclass={cs.LG}
}

Bayes error estimates

Error estimation is implemented three functions:

• hp_estimate for the Henze-Penrose estimator of the Bayes error rate. This can be used as:

  >>> import numpy as np
  >>> from smartsvm import hp_estimate
  >>> X1 = np.random.multivariate_normal([-1, 0], [[1, 0], [0, 1]], 100)
  >>> X2 = np.random.multivariate_normal([1, 0], [[1, 0], [0, 1]], 100)
  >>> hp_estimate(X1, X2) # with normalization
  >>> hp_estimate(X1, X2, normalize=False) # without normalization

• compute_error_graph and compute_ovr_error respectively compute the complete weighted graph of pairwise BER estimates or the One-vs-Rest BER for each class. They have a similar interface:

  >>> import numpy as np
  >>> from smartsvm import compute_error_graph, compute_ovr_error
  >>> from sklearn.datasets import load_digits
  >>> digits = load_digits(5)
  >>> n_samples = len(digits.images)
  >>> X = digits.images.reshape((n_samples, -1))
  >>> y = digits.target
  >>> G = compute_error_graph(X, y, n_jobs=2, normalize=True)
  >>> d = compute_ovr_error(X, y, normalize=True)

SmartSVM Classifier

SmartSVM is an adaptive hierarchical classifier which constructs a classification hierarchy based on the Henze-Penrose estimates of the Bayes error between each pair of classes. The classifier is build on top of Scikit-Learn and can be used in the exact same way as other sklearn classifiers:

>>> import numpy as np
>>> from smartsvm import SmartSVM
>>> from sklearn.datasets import load_digits
>>> digits = load_digits(10)
>>> n_samples = len(digits.images)
>>> X = digits.images.reshape((n_samples, -1))
>>> y = digits.target
>>> clf = SmartSVM()
>>> clf.fit(X, y)
>>> clf.predict(X)

By default, the SmartSVM classifier uses the Linear Support Vector Machine (LinearSVC) as the underlying binary classifier for each binary subproblem in the hierarchy. This can easily be changed with the binary_clf parameter to the class constructor, for instance:

>>> from sklearn.tree import DecisionTreeClassifier
>>> clf = SmartSVM(binary_clf=DecisionTreeClassifier)
>>> clf.fit(X, y)
>>> clf._get_binary()
DecisionTreeClassifier(class_weight=None, criterion='gini',
        max_depth=None, max_features=None, max_leaf_nodes=None,
        min_impurity_decrease=0.0, min_impurity_split=None,
        min_samples_leaf=1, min_samples_split=2,
        min_weight_fraction_leaf=0.0, presort=False, random_state=None,
        splitter='best')

You may optionally add parameters for the classifier through the clf_params parameter. This should be a dict with the parameters to the binary classifier, as follows:

>>> clf = SmartSVM(binary_clf=DecisionTreeClassifier, clf_params={'criterion': 'entropy'})
>>> clf.fit(X, y)
>>> clf._get_binary()
DecisionTreeClassifier(class_weight=None, criterion='entropy',
        max_depth=None, max_features=None, max_leaf_nodes=None,
        min_impurity_decrease=0.0, min_impurity_split=None,
        min_samples_leaf=1, min_samples_split=2,
        min_weight_fraction_leaf=0.0, presort=False, random_state=None,
        splitter='best')

Finally, it’s possible to retrieve probability estimates for the classes if the underlying classifier supports the predict_proba method:

>>> from sklearn.svm import SVC
>>> clf = SmartSVM(binary_clf=SVC, clf_params={"probabilities": True})
>>> clf.fit(X, y)
>>> prob = clf.predict_proba(X)
>>> import pandas as pd
>>> df = pd.DataFrame(prob)
>>> df
                 0             1             2      ...
0     9.999997e-01  1.716831e-18  2.677824e-13      ...
1     1.000000e-07  9.956408e-01  1.035589e-09      ...
2     2.595652e-05  1.452011e-02  9.722321e-01      ...

For more information about parameters to SmartSVM, see the API documentation here.