Support Vector Machines

Support vector machines (SVM) is a popular machine learning method with variants for classification, regression and distribution estimation that can learn a problem in a higher dimensional space through the use of a kernel trick. Integrated with Orange is a popular implementation by Chang and Lin, libsvm. Orange currently embeds version 8.1 of the library, which supports:

The support for the following kernel functions is provided by libsvm library:

See also LinearLearner for a fast linear SVM implementation.

SVMLearner

SVMLearner class constructs a SVMClassifier

Attributes

svm_type
Defines the type of SVM (can be SVMLearner.C_SVC (default), SVMLearner.Nu_SVC, SVMLearner.OneClass, SVMLearner.Epsilon_SVR, SVMLearner.Nu_SVR)
kernel_type
Defines the type of a kernel to use for learning (can be SVMLearner.RBF (default), SVMLearner.Linear, SVMLearner.Polynomial, SVMLearner.Sigmoid, SVMLearner.Custom)
degree
Kernel parameter (Polynomial) (default 3)
gamma
Kernel parameter (Polynomial/RBF/Sigmoid) (default 1.0/number_of_examples)
coef0
Kernel parameter (Polynomial/Sigmoid) (default 0)
kernelFunc
Function that will be called if kernel_type is SVMLearner.Custom. It must accept two orange.Example arguments and return a float.
C
C parameter for C_SVC, Epsilon_SVR, Nu_SVR
nu
Nu parameter for Nu_SVC, Nu_SVR and OneClass (default 0.5)
p
Epsilon in loss-function for Epsilon_SVR
cache_size
Cache memory size in MB (default 100)
eps
Tolerance of termination criterion (default 0.001)
shrinking
Determines whether to use shrinking heuristics (default True)
probability
Determines if a probability model should be build (default False)

SVMLearnerSparse

Same as above except that it learns from the examples mata attributes. Note that meta attributes dont need to be registerd with the dataset domain, or present in all the examples. Use this if you are using large sparse datasets.

SVMClassifier

Classifier used for classification, regression or distribution estimation (OneClass). In the later case the return value of the __call__ function can be 1.0 (positive case) or -1.0(negative case).

For a multiclass classification problem with k classes there are k*(k-1)/2 1class vs. 1class internal binary classifiers being build. The multiclass classification is then performed by a majority vote.

Attributes

examples
Holds the examples used for training
supportVectors
Holds the support vectors. They are listed in the order of their classes (i.e. they are grouped by the order of classes as they apear in the domains classVar.values)
nSV
Number of support vectors for each class (the same order as above)
rho
Constants in decision functions in the order of 1v2, 1v3, ... 1vsN, 2vs3, 2vs4, ...
coef
Coefficients for support vectors in decision functions (coef[nClass-1][nSupportVectors]). If k is the total number of classes then, for each support vector there are k-1 coefficients y*alpha where alpha are dual solution of the following two class problems: 1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k; and y=1 in first j-1 coefficients, y=-1 in the remaining k-j coefficients

Methods

getDecisionValues(example)
Return the decision values of all nClass*(nClass-1)/2 internal binary classifiers in the order of 1v2, 1v3, ... 1vsN, 2vs3, 2vs4, ...

Examples

>>> import orange >>> data=orange.ExampleTable("iris.tab") >>> l=orange.SVMLearner() >>> l.svm_type=orange.SVMLearner.Nu_SVC >>> l.nu=0.3 >>> l.probability=True >>> c=l(data) >>> for e in data: ... print e[-1], c(e), c(e, c.GetProbabilities) ... Iris-setosa Iris-setosa <0.971, 0.015, 0.014> Iris-setosa Iris-setosa <0.964, 0.019, 0.016> Iris-setosa Iris-setosa <0.968, 0.016, 0.016> ...

References

Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm