Zheng Zhao

Safe and efficient screening for sparse support vector machine

Sparse support vector machine (SVM) is a robust predictive model that can effectively remove noise and preserve signals. Like Lasso, it can efficiently learn a solution path based on a set of predefined parameters and therefore provides strong support for model selection. Sparse SVM has been successfully applied in a variety of data mining applications including text mining, bioinformatics, and image processing. The emergence of big-data analysis poses new challenges for model selection with large-scale data that consist of tens of millions samples and features. In this paper, a novel screening technique is proposed to accelerate model selection for l1-regularized l2-SVM and effectively improve its scalability. This technique can precisely identify inactive features in the optimal solution of a l1-regularized l2-SVM model and remove them before training. The technique makes use of the variational inequality and provides a closed-form solution for screening inactive features in different situations. Every feature that is removed by the screening technique is guaranteed to be inactive in the optimal solution. Therefore, when l1-regularized l2-SVM uses the features selected by the technique, it achieves exactly the same result as when it uses the full feature set. Because the technique can remove a large number of inactive features, it can greatly increase the efficiency of model selection for l1-regularized l2-SVM. Experimental results on five high-dimensional benchmark data sets demonstrate the power of the proposed technique.