Zhiji Yang

A Novel Twin Support-Vector Machine With Pinball Loss

Twin support-vector machine (TSVM), which generates two nonparallel hyperplanes by solving a pair of smaller-sized quadratic programming problems (QPPs) instead of a single larger-sized QPP, works faster than the standard SVM, especially for the large-scale data sets. However, the traditional TSVM adopts hinge loss which easily leads to its sensitivity of the noise and instability for resampling. To enhance the performance of the TSVM, we present a novel TSVM with the pinball loss (Pin-TSVM) which deals with the quantile distance and is less sensitive to noise points. We further investigate its properties, including the noise insensitivity, between-class distance maximization, and within-class scatter minimization. In addition, we compare our Pin-TSVM with the twin parametric-margin SVM and the SVM with the pinball loss in theory. Numerical experiments on a synthetic data set and 14 benchmark data sets with different noises demonstrate the feasibility and validity of our proposed method.

A maximum margin and minimum volume hyper-spheres machine with pinball loss for imbalanced data classification

The twin hyper-sphere support vector machine (THSVM) classifies two classes of samples via two hyper-spheres instead of a pair of nonparallel hyper-planes as in the conversional twin support vector machine (TSVM). Moreover THSVM avoids the matrix inverse operation when solving two dual quadratic programming problems (QPPs). However it cannot yield a desirable result when dealing with the imbalanced data classification. To improve the generalization performance, we propose a maximum margin and minimum volume hyper-spheres machine with pinball loss (Pin-M3HM) for the imbalanced data classification in this paper. The basic idea is to construct two hyper-spheres with different centers and radiuses in a sequential order. The first one contains as many examples in majority class as possible, and the second one covers minority class of examples as possible. Moreover the margin between two hyper-spheres is as large as possible. Besides, the pinball loss function is introduced into it to avoid the noise disturbance. Experimental results on 24 imbalanced datasets from the repositories of UCI and KEEL, and a real spectral dataset of Chinese grape wines indicate that our proposed Pin-M3HM yields a good generalization performance for the imbalanced data classification.

Safe Screening Rules for Accelerating Twin Support Vector Machine Classification

The twin support vector machine (TSVM) is widely used in classification problems, but it is not efficient enough for large-scale data sets. Furthermore, to get the optimal parameter, the exhaustive grid search method is applied to TSVM. It is very time-consuming, especially for multiparameter models. Although many techniques have been presented to solve these problems, all of them always affect the performance of TSVM to some extent. In this paper, we propose a safe screening rule (SSR) for linear-TSVM, and give a modified SSR (MSSR) for nonlinear TSVM, which contains multiple parameters. The SSR and MSSR can delete most training samples and reduce the scale of TSVM before solving it. Sequential versions of SSR and MSSR are further introduced to substantially accelerate the whole parameter tuning process. One important advantage of SSR and MSSR is that they are safe, i.e., we can obtain the same solution as the original problem by utilizing them. Experiments on eight real-world data sets and an imbalanced data set with different imbalanced ratios demonstrate the efficiency and safety of SSR and MSSR.

Structural least square twin support vector machine for classification

The least square twin support vector machine (LS-TSVM) obtains two non-parallel hyperplanes by directly solving two systems of linear equations instead of two quadratic programming problems (QPPs) as in the conventional twin support vector machine (TSVM), which makes the computational speed of LS-TSVM faster than that of the TSVM. However, LS-TSVM ignores the structural information of data which may contain some vital prior domain knowledge for training a classifier. In this paper, we apply the prior structural information of data into the LS-TSVM to build a better classifier, called the structural least square twin support vector machine (S-LSTSVM). Since it incorporates the data distribution information into the model, S-LSTSVM has good generalization performance. Furthermore, S-LSTSVM costs less time by solving two systems of linear equations compared with other existing methods based on structural information. Experimental results on twelve benchmark datasets demonstrate that our S-LSTSVM performs well. Finally, we apply it into Alzheimer’s disease diagnosis to further demonstrate the advantage of our algorithm.

Nonparallel least square support vector machine for classification

Nonparallel support vector machine based on one optimization problem (NSVMOOP) aims at finding two nonparallel hyper-planes by maximizing the intersection angle of their normal vectors w1 and w2. As maximum intersection angle preserves both compactness and separation of data, NSVMOOP yields good forecasting accuracy. However, as it solves one large quadratic programming problem (QPP), it costs high running time. In order to improve its learning speed, a novel nonparallel least square support vector machine (NLSSVM) is proposed in this paper. NLSSVM solves a linear system of equations instead of solving one large QPP. As both intersection angle and least square version are applied on our NLSSVM, it performs better generalization performance than other algorithms. Experimental results on twenty benchmark datasets demonstrate its validity.

v-twin support vector machine with Universum data for classification

A novel ν-twin support vector machine with Universum data (Uν

Asymmetric ν-twin support vector regression

\mathfrak {U}_{\nu }

KNN-based maximum margin and minimum volume hyper-sphere machine for imbalanced data classification

-TSVM) is proposed in this paper. Uν

Piecewise linear solution path for pinball twin support vector machine

\mathfrak {U}_{\nu }

A safe accelerative approach for pinball support vector machine classifier

-TSVM allows to incorporate the prior knowledge embedded in the unlabeled samples into the supervised learning. It aims to utilize these prior knowledge to improve the generalization performance. Different from the conventional U