Hongqiao Wang

Online chaotic time series prediction using unbiased composite kernel machine via Cholesky factorization

The kernel method has proved to be an effective machine learning tool in many fields. Support vector machines with various kernel functions may have different performances, as the kernels belong to two different types, the local kernels and the global kernels. So the composite kernel, which can bring more stable results and good precision in classification and regression, is an inevitable choice. To reduce the computational complexity of the kernel machine’s online modeling, an unbiased least squares support vector regression model with composite kernel is proposed. The bias item of LSSVR is eliminated by improving the form of structure risk in this model, and then the calculating method of the regression coefficients is greatly simplified. Simultaneously, through introducing the composite kernel to the LSSVM, the model can easily adapt to the irregular variation of the chaotic time series. Considering the real-time performance, an online learning algorithm based on Cholesky factorization is designed according to the characteristic of extended kernel function matrix. Experimental results indicate that the unbiased composite kernel LSSVR is effective and suitable for online time series with both the steep variations and the smooth variations, as it can well track the dynamic character of the series with good prediction precisions, better generalization and stability. The algorithm can also save much computation time comparing to those methods using matrix inversion, although there is a little more loss in time than that with the usage of single kernels.

An unbiased LSSVM model for classification and regression

Aiming at the common support vector machine’s biased disadvantage and computational complexity, an unbiased least squares support vector machine (LSSVM) model is proposed in this paper. The model eliminates the bias item of LSSVM by improving the form of structure risk, then the unbiased least squares support vector classifier and the unbiased least squares support vector regression are deduced. Based on this model, we design a new learning algorithm using Cholesky factorization according to the characteristic of kernel function matrix, in this way the calculation of Lagrangian multipliers is greatly simplified. Several experiments on diffenert datasets are carried out, including the common datasets classification, synthetic aperture radar image automatic target recognition and chaotic time series prediction. The experimental results of correct recognition rate and the fitting precision testify that the unbiased LSSVM model has good universal ability and fitting accuracy, better generalization capability and stability, and have a great improvement in learning speed.

Group based non-sparse localized multiple kernel learning algorithm for image classification

Multiple kernel learning is a new research focus in the field of kernel machine learning in recent years. Localized multiple kernel learning is a promising strategy for combining multiple features or kernels in terms of their discriminative power for different local space. In this paper, we proposed a group based non-sparse localized multiple kernel learning algorithm for image classification. There are two steps in our algorithm. In the first step, the samples are divided into groups according to a clustering algorithm. In the second step, the SVM model and local kernel weights are optimized by turns. By the process of clustering, both inter-cluster correlation and intra-cluster diversity are taken into concern. Since the Ip norm constraint is employed on the kernel weights, a non-sparse result of kernels is obtained. The performance of classifier is improved by adjusting the sparsity of kernels. The experiment on the synthetic data set shows that our method obtains a better decision boundary; the experiments on the image sets verify the improvement of classification accuracies and training speed.

Multiple feature fusion based image classification using a non-biased multi-scale kernel machine

Image target classification is an important branch of pattern recognition, especially the multi-class image classification is also a research focus for image recognition and retrieval. Aiming at the image characteristics of WANG dataset, a sub-dataset of Corel dataset, four effective feature extraction methods are studied in this paper, which are the color moment feature, the color distribution feature, the Fourier transform feature and the fractal dimension feature. On this basis, a non-biased multi-scale kernel least squares support vector classifier (LSSVC) is presented. Utilizing the non-biased LSSVC model, we can test and gain the optimal classification correct rate of each single feature, which can be used to determine the weight coefficients of multiple kernel learning. Ultimately, the different kinds of features and the multi-scale classifier can be organically fused. The multi-class image classification experiments show that the method has good generalizability, and can gain better classification precision.

A non-biased form of least squares support vector classifier and its fast online learning

As an effective learning technique based on structural risk minimization, SVM has been confirmed an useful tool in many machine learning fields. With the increase in application requirement for some real-time cases, such as fast prediction and pattern recognition, the online learning based on SVM gradually becomes a focus. But the common SVM has disadvantages in classifier’s bias and the computational complexity of online modeling, resulting in the reduction in classifier’s generality and the low learning speed. Therefore, an non-biased least square support vector classifier(LSSVC) model is proposed in this paper by improving the form of structure risk. Also, a fast online learning algorithm using Cholesky factorization is designed based on this model according to the characteristic of the non-biased kernel extended matrix in the model’s dynamic change process. In this way, the calculation of Lagrange multipliers is simplified, and the time of online learning is greatly reduced. Simulation results testify that the non-biased LSSVC has good universal applicability and better generalization capability, at the same time, the algorithm has a great improvement on learning speed.

Data-driven fault prediction and anomaly measurement for complex systems using support vector probability density estimation

Abstract To quantitatively monitor the state of complex system, a data-driven fault prediction and anomaly degree measurement method based on probability density estimation is studied in this paper. First, an anomaly index is introduced and defined to measure the anomaly degree of samples. Then By improving the form of constraint condition, a single slack factor multiple kernel support vector machine probability density estimation model is presented. As a result, the scale of object function and the solution number are all reduced, and the computational efficiency of the presented model is greatly enhanced. On the other hand, as the introduction of multiple kernel functions, a multiple kernel matrix with better data mapping performance is obtained, which can well solve the composite probability density estimation for uncoupled data. The simulation test shows that the presented model has higher estimation precision and speed. The experiments on complex system fault prediction also show that the system’s anomaly degree can be quantitatively and accurately measured by the anomaly index gained from the prediction results, which can effectively improve the fault prediction precision and increase the prediction advances.

Data-dependent multiple kernel learning algorithm based on soft-grouping

Abstract Multiple kernel learning strategy has emerged as a powerful tool because it can easily combine information from multiple data sources. However, learning an optimal kernel is still a challenging work and need to be further researched. In this paper, we propose a data-dependent multiple kernel learning algorithm based on soft-grouping (SC-DMKL). The core ideas of the SC-DMKL are twofold: (1) we take a soft-group process on the training samples to accommodate the correlation and the diversity of the samples; (2) alternatively optimize the kernel weights and the classifier to adaptively learn a data-dependent composite kernel. The final composite kernel is determined by the probability of samples falling to the groups and the kernel weights of these groups. Therefore, our method is actually a sample-specific MKL method with a soft restriction on the kernel weights. This restriction is actually the representation of the correlation of samples. The experiments on the synthetic dataset indicate that the kernel weights solved by our algorithm are more suitable for the characteristics of the datasets and the experiments on the real world datasets verify that the classification accuracies are improved.

Robust Automatic Target Recognition Algorithm for Large-Scene SAR Images and Its Adaptability Analysis on Speckle

Aiming at the multiple target recognition problems in large-scene SAR image with strong speckle, a robust full-process method from target detection, feature extraction to target recognition is studied in this paper. By introducing a simple 8-neighborhood orthogonal basis, a local multiscale decomposition method from the center of gravity of the target is presented. Using this method, an image can be processed with a multilevel sampling filter and the target’s multiscale features in eight directions and one low frequency filtering feature can be derived directly by the key pixels sampling. At the same time, a recognition algorithm organically integrating the local multiscale features and the multiscale wavelet kernel classifier is studied, which realizes the quick classification with robustness and high accuracy for multiclass image targets. The results of classification and adaptability analysis on speckle show that the robust algorithm is effective not only for the MSTAR Moving and Stationary Target Automatic Recognition target chips but also for the automatic target recognition of multiclass/multitarget in large-scene SAR image with strong speckle; meanwhile, the method has good robustness to target’s rotation and scale transformation.

Visual-Cognition-Driven SAR Multiple Targets Robust Feature Extraction, Recognition and Tracking

Aiming at the multiple targets recognition and tracking in SAR images, a robust feature extraction method and a combined recognition and tracking method for multi-class slow-moving targets based on visual cognition is presented in this paper. To obtain robust feature and high classification precision, a local multi-resolution analysis and feature extraction method based on the visual attention mechanism and a multiple kernel classifier is studied, which realizes the quick classification with high accuracy for multi-class image targets. According to the recognition result and the corresponding relationship of targets in the adjacent frames, the targets’ motion parameters are estimated utilizing the unscented Kalman filter (UKF) based on the “what” and “where” pathways information processing mechanism. As a result, the high performance tracking of multi-class slow-moving targets in complicated background is realized. The simulation results show that the feature extraction and recognition method has good robustness and high classification correct ratio, the combining recognition and tracking method also has high location precision.

Classification Probability Estimation Based Multi-Class Image Retrieval

Aiming at multi-class large-scale image retrieval problem, a new image retrieval method based on classification probability estimation is proposed according to the thinking named “Classification First, Retrieval Later”. According to the method, the image features are effectively fused using a composite kernel method first, and a composite kernel classifier with higher classification precision is designed. The optimal coefficients of the classifier are also obtained utilizing the classification result with small-amount image samples. Second, complete the classification probability estimation for the testing images using the composite machine. Third, realize the image retrieval based on the classification probability estimation values. In the experiments with multi-class large-scale image dataset, it is confirmed that the presented method can achieve better retrieval precision. Moreover, the generalization performance without prior knowledge is also studied.