----------------------------------
--- Python interface of LIBSVM ---
----------------------------------
Table of Contents
=================
- Introduction
- Installation via PyPI
- Installation via Sources
- Quick Start
- Quick Start with Scipy
- Design Description
- Data Structures
- Utility Functions
- Additional Information
Introduction
============
Python (http://www.python.org/) is a programming language suitable for rapid
development. This tool provides a simple Python interface to LIBSVM, a library
for support vector machines (http://www.csie.ntu.edu.tw/~cjlin/libsvm). The
interface is very easy to use as the usage is the same as that of LIBSVM. The
interface is developed with the built-in Python library "ctypes."
Installation via PyPI
=====================
To install the interface from PyPI, execute the following command:
> pip install -U libsvm-official
Installation via Sources
========================
Alternatively, you may install the interface from sources by
generating the LIBSVM shared library.
Depending on your use cases, you can choose between local-directory
and system-wide installation.
- Local-directory installation:
On Unix systems, type
> make
This generates a .so file in the LIBSVM main directory and you
can run the interface in the current python directory.
For Windows, the shared library libsvm.dll is ready in the
directory `..\windows' and you can directly run the interface in
the current python directory. You can copy libsvm.dll to the
system directory (e.g., `C:\WINDOWS\system32\') to make it
system-widely available. To regenerate libsvm.dll, please
follow the instruction of building Windows binaries in LIBSVM
README.
- System-wide installation:
Type
> pip install -e .
or
> pip install --user -e .
The option --user would install the package in the home directory
instead of the system directory, and thus does not require the
root privilege.
Please note that you must keep the sources after the installation.
For Windows, to run the above command, Microsoft Visual C++ and
other tools are needed.
In addition, DON'T use the following FAILED commands
> python setup.py install (failed to run at the python directory)
> pip install .
Quick Start
===========
"Quick Start with Scipy" is in the next section.
There are two levels of usage. The high-level one uses utility
functions in svmutil.py and commonutil.py (shared with LIBLINEAR and
imported by svmutil.py). The usage is the same as the LIBSVM MATLAB
interface.
>>> from libsvm.svmutil import *
# Read data in LIBSVM format
>>> y, x = svm_read_problem('../heart_scale')
>>> m = svm_train(y[:200], x[:200], '-c 4')
>>> p_label, p_acc, p_val = svm_predict(y[200:], x[200:], m)
# Construct problem in python format
# Dense data
>>> y, x = [1,-1], [[1,0,1], [-1,0,-1]]
# Sparse data
>>> y, x = [1,-1], [{1:1, 3:1}, {1:-1,3:-1}]
>>> prob = svm_problem(y, x)
>>> param = svm_parameter('-t 0 -c 4 -b 1')
>>> m = svm_train(prob, param)
# Precomputed kernel data (-t 4)
# Dense data
>>> y, x = [1,-1], [[1, 2, -2], [2, -2, 2]]
# Sparse data
>>> y, x = [1,-1], [{0:1, 1:2, 2:-2}, {0:2, 1:-2, 2:2}]
# isKernel=True must be set for precomputed kernel
>>> prob = svm_problem(y, x, isKernel=True)
>>> param = svm_parameter('-t 4 -c 4 -b 1')
>>> m = svm_train(prob, param)
# For the format of precomputed kernel, please read LIBSVM README.
# Other utility functions
>>> svm_save_model('heart_scale.model', m)
>>> m = svm_load_model('heart_scale.model')
>>> p_label, p_acc, p_val = svm_predict(y, x, m, '-b 1')
>>> ACC, MSE, SCC = evaluations(y, p_label)
# Getting online help
>>> help(svm_train)
The low-level use directly calls C interfaces imported by svm.py. Note that
all arguments and return values are in ctypes format. You need to handle them
carefully.
>>> from libsvm.svm import *
>>> prob = svm_problem([1,-1], [{1:1, 3:1}, {1:-1,3:-1}])
>>> param = svm_parameter('-c 4')
>>> m = libsvm.svm_train(prob, param) # m is a ctype pointer to an svm_model
# Convert a Python-format instance to svm_nodearray, a ctypes structure
>>> x0, max_idx = gen_svm_nodearray({1:1, 3:1})
>>> label = libsvm.svm_predict(m, x0)
Quick Start with Scipy
======================
Make sure you have Scipy installed to proceed in this section.
If numba (http://numba.pydata.org) is installed, some operations will be much faster.
There are two levels of usage. The high-level one uses utility functions
in svmutil.py and the usage is the same as the LIBSVM MATLAB interface.
>>> import numpy as np
>>> import scipy
>>> from libsvm.svmutil import *
# Read data in LIBSVM format
>>> y, x = svm_read_problem('../heart_scale', return_scipy = True) # y: ndarray, x: csr_matrix
>>> m = svm_train(y[:200], x[:200, :], '-c 4')
>>> p_label, p_acc, p_val = svm_predict(y[200:], x[200:, :], m)
# Construct problem in Scipy format
# Dense data: numpy ndarray
>>> y, x = np.asarray([1,-1]), np.asarray([[1,0,1], [-1,0,-1]])
# Sparse data: scipy csr_matrix((data, (row_ind, col_ind))
>>> y, x = np.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 1, -1, -1], ([0, 0, 1, 1], [0, 2, 0, 2])))
>>> prob = svm_problem(y, x)
>>> param = svm_parameter('-t 0 -c 4 -b 1')
>>> m = svm_train(prob, param)
# Precomputed kernel data (-t 4)
# Dense data: numpy ndarray
>>> y, x = np.asarray([1,-1]), np.asarray([[1,2,-2], [2,-2,2]])
# Sparse data: scipy csr_matrix((data, (row_ind, col_ind))
>>> y, x = np.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 2, -2, 2, -2, 2], ([0, 0, 0, 1, 1, 1], [0, 1, 2, 0, 1, 2])))
# isKernel=True must be set for precomputed kernel
>>> prob = svm_problem(y, x, isKernel=True)
>>> param = svm_parameter('-t 4 -c 4 -b 1')
>>> m = svm_train(prob, param)
# For the format of precomputed kernel, please read LIBSVM README.
# Apply data scaling in Scipy format
>>> y, x = svm_read_problem('../heart_scale', return_scipy=True)
>>> scale_param = csr_find_scale_param(x, lower=0)
>>> scaled_x = csr_scale(x, scale_param)
# Other utility functions
>>> svm_save_model('heart_scale.model', m)
>>> m = svm_load_model('heart_scale.model')
>>> p_label, p_acc, p_val = svm_predict(y, x, m, '-b 1')
>>> ACC, MSE, SCC = evaluations(y, p_label)
# Getting online help
>>> help(svm_train)
The low-level use directly calls C interfaces imported by svm.py. Note that
all arguments and return values are in ctypes format. You need to handle them
carefully.
>>> from libsvm.svm import *
>>> prob = svm_problem(np.asarray([1,-1]), scipy.sparse.csr_matrix(([1, 1, -1, -1], ([0, 0, 1, 1], [0, 2, 0, 2]))))
>>> param = svm_parameter('-c 4')
>>> m = libsvm.svm_train(prob, param) # m is a ctype pointer to an svm_model
# Convert a tuple of ndarray (index, data) to feature_nodearray, a ctypes structure
# Note that index starts from 0, though the following example will be changed to 1:1, 3:1 internally
>>> x0, max_idx = gen_svm_nodearray((np.asarray([0,2]), np.asarray([1,1])))
>>> label = libsvm.svm_predict(m, x0)
Design Description
==================
There are two files svm.py and svmutil.py, which respectively correspond to
low-level and high-level use of the interface.
In svm.py, we adopt the Python built-in library "ctypes," so that
Python can directly access C structures and interface functions defined
in svm.h.
While advanced users can use structures/functions in svm.py, to
avoid handling ctypes structures, in svmutil.py we provide some easy-to-use
functions. The usage is similar to LIBSVM MATLAB interface.
Data Structures
===============
Four data structures derived from svm.h are svm_node, svm_problem, svm_parameter,
and svm_model. They all contain fields with the same names in svm.h. Access
these fields carefully because you directly use a C structure instead of a
Python object. For svm_model, accessing the field directly is not recommanded.
Programmers should use the interface functions or methods of svm_model class
in Python to get the values. The following description introduces additional
fields and methods.
Before using the data structures, execute the following command to load the
LIBSVM shared library:
>>> from libsvm.svm import *
- class svm_node:
Construct an svm_node.
>>> node = svm_node(idx, val)
idx: an integer indicates the feature index.
val: a float indicates the feature value.
Show the index and the value of a node.
>>> print(node)
- Function: gen_svm_nodearray(xi [,feature_max=None [,isKernel=False]])
Generate a feature vector from a Python list/tuple/dictionary, numpy ndarray or tuple of (index, data):
>>> xi_ctype, max_idx = gen_svm_nodearray({1:1, 3:1, 5:-2})
xi_ctype: the returned svm_nodearray (a ctypes structure)
max_idx: the maximal feature index of xi
feature_max: if feature_max is assigned, features with indices larger than
feature_max are removed.
isKernel: if isKernel == True, the list index starts from 0 for precomputed
kernel. Otherwise, the list index starts from 1. The default
value is False.
- class svm_problem:
Construct an svm_problem instance
>>> prob = svm_problem(y, x)
y: a Python list/tuple/ndarray of l labels (type must be int/double).
x: 1. a list/tuple of l training instances. Feature vector of
each training instance is a list/tuple or dictionary.
2. an l * n numpy ndarray or scipy spmatrix (n: number of features).
Note that if your x contains sparse data (i.e., dictionary), the internal
ctypes data format is still sparse.
For pre-computed kernel, the isKernel flag should be set to True:
>>> prob = svm_problem(y, x, isKernel=True)
Please read LIBSVM README for more details of pre-computed kernel.
- class svm_parameter:
Construct an svm_parameter instance
>>> param = svm_parameter('training_options')
If 'training_options' is empty, LIBSVM default values are applied.
Set param to LIBSVM default values.
>>> param.set_to_default_values()
Parse a string of options.
>>> param.parse_options('training_options')
Show values of parameters.
>>> print(param)
- class svm_model:
There are two ways to obtain an instance of svm_model:
>>> model = svm_train(y, x)
>>> model = svm_load_model('model_file_name')
Note that the returned structure of interface functions
libsvm.svm_train and libsvm.svm_load_model is a ctypes pointer of
svm_model, which is different from the svm_model object returned
by svm_train and svm_load_model in svmutil.py. We provide a
function toPyModel for the conversion:
>>> model_ptr = libsvm.svm_train(prob, param)
>>> model = toPyModel(model_ptr)
If you obtain a model in a way other than the above approaches,
handle it carefully to avoid memory leak or segmentation fault.
Some interface functions to access LIBSVM models are wrapped as
members of the class svm_model:
>>> svm_type = model.get_svm_type()
>>> nr_class = model.get_nr_class()
>>> svr_probability = model.get_svr_probability()
>>> class_labels = model.get_labels()
>>> sv_indices = model.get_sv_indices()
>>> nr_sv = model.get_nr_sv()
>>> is_prob_model = model.is_probability_model()
>>> support_vector_coefficients = model.get_sv_coef()
>>> support_vectors = model.get_SV()
Utility Functions
=================
To use utility functions, type
>>> from libsvm.svmutil import *
The above command loads
svm_train() : train an SVM model
svm_predict() : predict testing data
svm_read_problem() : read the data from a LIBSVM-format file.
svm_load_model() : load a LIBSVM model.
svm_save_model() : save model to a file.
evaluations() : evaluate prediction results.
csr_find_scale_param() : find scaling parameter for data in csr format.
csr_scale() : apply data scaling to data in csr format.
- Function: svm_train
There are three ways to call svm_train()
>>> model = svm_train(y, x [, 'training_options'])
>>> model = svm_train(prob [, 'training_options'])
>>> model = svm_train(prob, param)
y: a list/tuple/ndarray of l training labels (type must be int/double).
x: 1. a list/tuple of l training instances. Feature vector of
each training instance is a list/tuple or dictionary.
2. an l * n numpy ndarray or scipy spmatrix (n: number of features).
training_options: a string in the same form as that for LIBSVM command
mode.
prob: an svm_problem instance generated by calling
svm_problem(y, x).
For pre-computed kernel, you should use
svm_problem(y, x, isKernel=True)
param: an svm_parameter instance generated by calling
svm_parameter('training_options')
model: the returned svm_model instance. See svm.h for details of this
structure. If '-v' is specified, cross validation is
conducted and the returned model is just a scalar: cross-validation
accuracy for classification and mean-squared error for regression.
To train the same data many times with different
parameters, the second and the third ways should be faster..
Examples:
>>> y, x = svm_read_problem('../heart_scale')
>>> prob = svm_problem(y, x)
>>> param = svm_parameter('-s 3 -c 5 -h 0')
>>> m = svm_train(y, x, '-c 5')
>>> m = svm_train(prob, '-t 2 -c 5')
>>> m = svm_train(prob, param)
>>> CV_ACC = svm_train(y, x, '-v 3')
- Function: svm_predict
To predict testing data with a model, use
>>> p_labs, p_acc, p_vals = svm_predict(y, x, model [,'predicting_options'])
y: a list/tuple/ndarray of l true labels (type must be int/double).
It is used for calculating the accuracy. Use [] if true labels are
unavailable.
x: 1. a list/tuple of l training instances. Feature vector of
each training instance is a list/tuple or dictionary.
2. an l * n numpy ndarray or scipy spmatrix (n: number of features).
predicting_options: a string of predicting options in the same format as
that of LIBSVM.
model: an svm_model instance.
p_labels: a list of predicted labels
p_acc: a tuple including accuracy (for classification), mean
squared error, and squared correlation coefficient (for
regression).
p_vals: a list of decision values or probability estimates (if '-b 1'
is specified). If k is the number of classes in training data,
for decision values, each element includes results of predicting
k(k-1)/2 binary-class SVMs. For classification, k = 1 is a
special case. Decision value [+1] is returned for each testing
instance, instead of an empty list.
For probabilities, each element contains k values indicating
the probability that the testing instance is in each class.
For one-class SVM, the list has two elements indicating the
probabilities of normal instance/outlier.
Note that the order of classes is the same as the 'model.label'
field in the model structure.
Example:
>>> m = svm_train(y, x, '-c 5')
>>> p_labels, p_acc, p_vals = svm_predict(y, x, m)
- Functions: svm_read_problem/svm_load_model/svm_save_model
See the usage by examples:
>>> y, x = svm_read_problem('data.txt')
>>> m = svm_load_model('model_file')
>>> svm_save_model('model_file', m)
- Function: evaluations
Calculate some evaluations using the true values (ty) and the predicted
values (pv):
>>> (ACC, MSE, SCC) = evaluations(ty, pv, useScipy)
ty: a list/tuple/ndarray of true values.
pv: a list/tuple/ndarray of predicted values.
useScipy: convert ty, pv to ndarray, and use scipy functions to do the evaluation
ACC: accuracy.
MSE: mean squared error.
SCC: squared correlation coefficient.
- Function: csr_find_scale_parameter/csr_scale
Scale data in csr format.
>>> param = csr_find_scale_param(x [, lower=l, upper=u])
>>> x = csr_scale(x, param)
x: a csr_matrix of data.
l: x scaling lower limit; default -1.
u: x scaling upper limit; default 1.
The scaling process is: x * diag(coef) + ones(l, 1) * offset'
param: a dictionary of scaling parameters, where param['coef'] = coef and param['offset'] = offset.
coef: a scipy array of scaling coefficients.
offset: a scipy array of scaling offsets.
Additional Information
======================
This interface was originally written by Hsiang-Fu Yu from Department of Computer
Science, National Taiwan University. If you find this tool useful, please
cite LIBSVM as follows
Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support
vector machines. ACM Transactions on Intelligent Systems and
Technology, 2:27:1--27:27, 2011. Software available at
http://www.csie.ntu.edu.tw/~cjlin/libsvm
For any question, please contact Chih-Jen Lin <cjlin@csie.ntu.edu.tw>,
or check the FAQ page:
http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html
