Lingfeng Niu

Learning Using Privileged Information with L-1 Support Vector Machine

In the process of human learning, teachers always play an important role. However, for most of the existing machine learning method, the role of teachers is seldom considered. Recently, Vapnik introduced an advanced learning paradigm called Learning Using Privileged Information(LUPI) to include the elements of human teaching in machine learning. Through theoretical analysis and numerical experiments, the superiority of LUPI over the classical learning paradigm has received preliminary proof. In this paper, on the basis of existing work for LUPI, we introduce the privileged information into the modeling of L-1 support vector machine(SVM). Compared with the existing research of LUPI with L-2 SVM, the new method has the advantage of spending less time on tuning model parameters and the additional benefits of performing feature selection in the training process. Experiments on the digit recognition problem validate the effectiveness of our method.

A Simple Regularized Multiple Criteria Linear Programs for Binary Classification

Abstract Optimization is an important tool in computational finance and business intelligence. Multiple criteria mathematical pro- gram(MCMP), which is concerned with mathematical optimization problems involving more than one objective function to be optimized simultaneously, is one of the ways of utilizing optimization techniques. Due to the existence of multiple objec- tives, MCMPs are usually difficult to be optimized. In fact, for a nontrivial MCMP, there does not exist a single solution that optimizes all the objectives at the same time. In practice, many methods convert the original MCMP into a single-objective program and solve the obtained scalarized optimization problem. If the values of scalarization parameters, which measure the trade-offs between the conflicting objectives, are not chosen carefully, the converted single-objective optimization problem may be not solvable. Therefore, to make sure MCMP always can be solved successfully, heuristic search and expert knowledge for deciding the value of scalarization parameters are always necessary, which is not an easy task and limits the applications of MCMP to some extend. In this paper, we take the multiple criteria linear program(MCLP) for binary classification as the example and discuss how to modified the formulation of MCLP directly to guarantee the solvability. In details, we propose adding a quadratic regularization term into the converted single-objective linear program. The new regularized formulation does not only overcomes some defects of the original scalarized problem in modeling, it also can be shown in theory that the finite optimal solutions always exist. To test the performance of the proposed method, we compare our algorithm with sever- al state-of-the-art algorithms for binary classification on several different kinds of datasets. Preliminary experimental results demonstrate the effectiveness of our regularization method.

Feature Selection With ℓ2,1-2 Regularization

Feature selection aims to select a subset of features from high-dimensional data according to a predefined selecting criterion. Sparse learning has been proven to be a powerful technique in feature selection. Sparse regularizer, as a key component of sparse learning, has been studied for several years. Although convex regularizers have been used in many works, there are some cases where nonconvex regularizers outperform convex regularizers. To make the process of selecting relevant features more effective, we propose a novel nonconvex sparse metric on matrices as the sparsity regularization in this paper. The new nonconvex regularizer could be written as the difference of the

Entity Disambiguation with Textual and Connection Information

\ell _{2,1}

Pedestrian detection based on the privileged information

norm and the Frobenius (

Large-scale Nonparallel Support Vector Ordinal Regression Solver

\ell _{2,2}

Kernel based simple regularized multiple criteria linear program for binary classification and regression

) norm, which is named the

A First-Order Decomposition Algorithm for Training Bound-Constrained Support Vector Machines

\ell _{2,1-2}

Robust Unsupervised Feature Learning from Time-Series

. To find the solution of the resulting nonconvex formula, we design an iterative algorithm in the framework of ConCave-Convex Procedure (CCCP) and prove its strong global convergence. An adopted alternating direction method of multipliers is embedded to solve the sequence of convex subproblems in CCCP efficiently. Using the scaled cluster indictors of data points as pseudolabels, we also apply

A Simple Decomposition Alternating Direction Method for Matrix Completion

\ell _{2,1-2}