Kernel Path for ν-Support Vector Classification.

Abstract

It is well known that the performance of a kernel method is highly dependent on the choice of kernel parameter. However, existing kernel path algorithms are limited to plain support vector machines (SVMs), which has one equality constraint. It is still an open question to provide a kernel path algorithm to ν-support vector classification (ν-SVC) with more than one equality constraint. Compared with plain SVM, ν-SVC has the advantage of using a regularization parameter ν for controlling the number of support vectors and margin errors. To address this problem, in this article, we propose a kernel path algorithm (KPνSVC) to trace the solutions of ν-SVC exactly with respect to the kernel parameter. Specifically, we first provide an equivalent formulation of ν-SVC with two equality constraints, which can avoid possible conflicts during tracing the solutions of ν-SVC. Based on this equivalent formulation of ν-SVC, we propose the KPνSVC algorithm to trace the solutions with respect to the kernel parameter. However, KPνSVC traces nonlinear solutions of kernel method rather than the errors of loss function, and it is still a challenge to provide the algorithm that guarantees to find the global optimal model. To address this challenging problem, we extend the classical error path algorithm to the nonlinear kernel solution paths and propose a new kernel error path (KEP) algorithm that ensures to find the global optimal kernel parameter by minimizing the cross validation error. We also provide the finite convergence analysis and computational complexity analysis to KPνSVC and KEP. Extensive experimental results on a variety of benchmark datasets not only verify the effectiveness of KPνSVC but also show the advantage of applying KEP to select the optimal kernel parameter.