Xi Zhao

Prediction of Customer Attrition of Commercial Banks based on SVM Model

Abstract Currently, Chinese commercial banks are facing triple tremendous pressure, including financial disintermediation, interest rate marketization and Internet finance. Meanwhile, increasing financial consumption demand of customers further intensifies the competition among commercial banks. To increase their profits for continuing operations and enhance the core competitiveness, commercial banks must avoid the loss of customers while acquiring new customers. This paper discusses commercial bank customer churn prediction based on SVM model, and uses random sampling method to improve SVM model, considering the imbalance characteristics of customer data sets. The results show that this method can effectively enhance the prediction accuracy of the selected model.

Feature Selection with Attributes Clustering by Maximal Information Coefficient

Abstract Feature selection is usually a separate procedure which can not benefit from result of the data exploration. In this paper, we propose a unsupervised feature selection method which could reuse a specific data exploration result. Furthermore, our algorithm follows the idea of clustering attributes and combines two state-of-the-art data analyzing methods, thats maximal information coefficient and affinity propagation. Classification problems with different classifiers were tested to validation our method and others. Data experiments result exhibits our unsupervised algorithm is comparable with classical feature selection methods and even outperforms some supervised learning algorithms. Data simulation with one credit dataset of our own from a bank of China shows the capability of our method for real world application.

Local and Global Regularized Twin SVM

The generalized eigenvalue proximal support vector machine (GEPSVM) and twin support vector machine (TWSVM) was proposed by Mangasarian and Jayadeva respectively, which aroused the interest of academia for its less computation cost and better generalization ability. They use the nonparallel hyperplane classifiers to solve the classification problem. Different from traditionally local or global TWSVM methods, a new Twin SVM algorithm called Local and Global Regularized Twin SVM (TWSVMLG) is proposed in this paper. A global regularizer was imposed across local models to smooth the data labels predicted by those local classifiers and avoid overfitting risk for the local classifiers. The classifier could get stronger discriminating ability when exploring local and global information than traditional algorithms. Finally some experimental results are presented to show the effectiveness of our algorithm.

A Simple Regularized Multiple Criteria Linear Programs for Binary Classification

Abstract Optimization is an important tool in computational finance and business intelligence. Multiple criteria mathematical pro- gram(MCMP), which is concerned with mathematical optimization problems involving more than one objective function to be optimized simultaneously, is one of the ways of utilizing optimization techniques. Due to the existence of multiple objec- tives, MCMPs are usually difficult to be optimized. In fact, for a nontrivial MCMP, there does not exist a single solution that optimizes all the objectives at the same time. In practice, many methods convert the original MCMP into a single-objective program and solve the obtained scalarized optimization problem. If the values of scalarization parameters, which measure the trade-offs between the conflicting objectives, are not chosen carefully, the converted single-objective optimization problem may be not solvable. Therefore, to make sure MCMP always can be solved successfully, heuristic search and expert knowledge for deciding the value of scalarization parameters are always necessary, which is not an easy task and limits the applications of MCMP to some extend. In this paper, we take the multiple criteria linear program(MCLP) for binary classification as the example and discuss how to modified the formulation of MCLP directly to guarantee the solvability. In details, we propose adding a quadratic regularization term into the converted single-objective linear program. The new regularized formulation does not only overcomes some defects of the original scalarized problem in modeling, it also can be shown in theory that the finite optimal solutions always exist. To test the performance of the proposed method, we compare our algorithm with sever- al state-of-the-art algorithms for binary classification on several different kinds of datasets. Preliminary experimental results demonstrate the effectiveness of our regularization method.

Kernel based simple regularized multiple criteria linear program for binary classification and regression

Handling data classification and regression problems through linear hyperplane is a naive and simple idea. In this paper, inspired by the idea of multiple criteria linear programs (MCLP) and multiple criteria quadratic programs (MCQP), we proposed a novel method for binary classification and regression problem. There are two main advantages for the proposed approach. One is that both of these two models guarantee the existence of feasible solutions when the model parameters were chosen properly. The other is that nonlinear patterns could be handled and captured by introducing kernel function into MCLP framework with a more natural way than previous work. Various classical approaches and datasets were evaluated in our experiments, and the result on both toy and real world data demonstrate the correctness and effectiveness of our proposed methods.

Two New Decomposition Algorithms for Training Bound-Constrained Support Vector Machines*

Abstract Bound-constrained Support Vector Machine(SVM) is one of the stateof- art model for binary classification. The decomposition method is currently one of the major methods for training SVMs, especially when the nonlinear kernel is used. In this paper, we proposed two new decomposition algorithms for training bound-constrained SVMs. Projected gradient algorithm and interior point method are combined together to solve the quadratic subproblem effciently. The main difference between the two algorithms is the way of choosing working set. The first one only uses first order derivative information of the model for simplicity. The second one incorporate part of second order information into the process of working set selection, besides the gradient. Both algorithms are proved to be global convergent in theory. New algorithms is compared with the famous package BSVM. Numerical experiments on several public data sets validate the effciency of the proposed methods.

A First-Order Decomposition Algorithm for Training Bound-Constrained Support Vector Machines

We present a new decomposition algorithm for training bound-constrained Support Vector Machines in this paper. When selecting indices into the working set, only first order derivative information of the objective function in the optimization model is required. Therefore, the resulting working set selection strategy is simple and can be implemented easily. The new algorithm is proved to be global convergent in theory. New algorithm is compared with the state-of-art package BSVM. Numerical experiments on several public data sets also validate the effectiveness and efficiency of the proposed method.

Kernel Based Simple Regularized Multiple Criteria Linear Programs for Binary Classification

Binary classification is the simplest case for classification problem. In this paper, we proposed a new kernel based regularized multiple criteria linear programs for binary classification. Modeling capability of original regularized model were improved by introducing nonlinear kernel, which made the model could identify nonlinear patterns in data. Experiments result showed our model is comparable with classical binary classifiers and suitable for real world application.

A Simple Decomposition Alternating Direction Method for Matrix Completion

Matrix completion(MC), which is to recover a data matrix from a sampling of its entries, arises in many applications. In this work, we consider find the solutions of the MC problems by solving a series of fixed rank problems. For the fixed rank problems, variables are divided into two parts naturally based on matrix factorization and a simple alternative direction method framework is proposed. For each fixed rank problem, the solving process of each part of variables can be further converted into a series of relative small scale independent linear equations systems. Based on these observations, we design a decomposition alternative direction method for the MC problem. To test the performance of the new method, we implement our method in Matlab(with a few C/Mex functions) and compare it with several state-of-the-art solvers for the MC problem. Preliminary experimental results indeed demonstrate the effectiveness and efficiency of our method.