Linkai Luo

Improving the Computational Efficiency of Recursive Cluster Elimination for Gene Selection

The gene expression data are usually provided with a large number of genes and a relatively small number of samples, which brings a lot of new challenges. Selecting those informative genes becomes the main issue in microarray data analysis. Recursive cluster elimination based on support vector machine (SVM-RCE) has shown the better classification accuracy on some microarray data sets than recursive feature elimination based on support vector machine (SVM-RFE). However, SVM-RCE is extremely time-consuming. In this paper, we propose an improved method of SVM-RCE called ISVM-RCE. ISVM-RCE first trains a SVM model with all clusters, then applies the infinite norm of weight coefficient vector in each cluster to score the cluster, finally eliminates the gene clusters with the lowest score. In addition, ISVM-RCE eliminates genes within the clusters instead of removing a cluster of genes when the number of clusters is small. We have tested ISVM-RCE on six gene expression data sets and compared their performances with SVM-RCE and linear-discriminant-analysis-based RFE (LDA-RFE). The experiment results on these data sets show that ISVM-RCE greatly reduces the time cost of SVM-RCE, meanwhile obtains comparable classification performance as SVM-RCE, while LDA-RFE is not stable.

Structural damage detection based on posteriori probability support vector machine and Dempster-Shafer evidence theory

Proposed a stable structural damage detection method based on information fusion.Improved the accuracy and stability of the damage detection than using single sensor.Solved the bad impact of any sensor failure and give more robust detection results. An intelligent detection method is proposed in this paper to enrich the study of applying machine learning and data mining techniques to building structural damage identification. The proposed method integrates the multi-sensory data fusion and classifier ensemble to detect the location and extent of the damage. First, the wavelet package analysis is used to transform the original vibration acceleration signal into energy features. Then the posteriori probability support vector machines (PPSVM) and the Dempster-Shafer (DS) evidence theory are combined to identify the damage. Empirical study on a benchmark structure model shows that, compared with popular data mining approaches, the proposed method can provide more accurate and stable detection results. Furthermore, this paper compares the detection performance of the information fusion at different levels. The experimental analysis demonstrates that the proposed method with the fusion at the decision level can make good use of multi-sensory information and is more robust in practice.

Using the Maximum Between-Class Variance for Automatic Gridding of cDNA Microarray Images

Gridding is the first and most important step to separate the spots into distinct areas in microarray image analysis. Human intervention is necessary for most gridding methods, even if some so-called fully automatic approaches also need preset parameters. The applicability of these methods is limited in certain domains and will cause variations in the gene expression results. In addition, improper gridding, which is influenced by both the misalignment and high noise level, will affect the high throughput analysis. In this paper, we have presented a fully automatic gridding technique to break through the limitation of traditional mathematical morphology gridding methods. First, a preprocessing algorithm was applied for noise reduction. Subsequently, the optimal threshold was gained by using the improved Otsu method to actually locate each spot. In order to diminish the error, the original gridding result was optimized according to the heuristic techniques by estimating the distribution of the spots. Intensive experiments on six different data sets indicate that our method is superior to the traditional morphology one and is robust in the presence of noise. More importantly, the algorithm involved in our method is simple. Furthermore, human intervention and parameters presetting are unnecessary when the algorithm is applied in different types of microarray images.

Methods of forward feature selection based on the aggregation of classifiers generated by single attribute

Compared to backward feature selection (BFS) method in gene expression data analysis, forward feature selection (FFS) method can obtain an expected feature subset with less iteration. However, the number of FFS method is considerably less than that of BFS method. More efficient FFS methods need to be developed. In this paper, two FFS methods based on the pruning of the classifier ensembles generated by single attribute are proposed for gene selection. The main contributions are as follows: (1) a new loss function, p-insensitive loss function, is proposed to overcome the disadvantage of the margin Euclidean distance loss function in the pruning of classifier ensembles; (2) two FFS methods based on the margin Euclidean distance loss function and the p-insensitive loss function, named as FFS-ACSA1 and FFS-ACSA2 respectively, are proposed; (3) the comparison experiments on four gene expression datasets show that FFS-ACSA2 obtains the best results among three FFS methods (i.e. signal-to-noise ratio (SNR), FFS-ACSA1 and FFS-ACSA2), and is competitive to the famous support vector machine-based recursive feature elimination (SVM-RFE), while FFS-ACSA1 is unstable.

Structural damage detection method based on random forests and data fusion

A structural damage detection method by integrating data fusion and random forests was proposed. The original acceleration signals were translated into energy features by wavelet packet decomposition. Then the processed energy features were fused into new energy features by data fusion. This can further enlarge the differences among all types of damages. Finally, random forests as an effective classifier was used to detect the multiclass damage. Numerical study on the benchmark model and an eight-storey steel shear frame structure model was carried out to validate the accuracy of the proposed damage detection method. The experiment results indicate that the damage detection method based on random forests and data fusion can improve damage detection accuracy in comparison with random forests alone, support vector machine alone, and support vector machine and data fusion techniques. Moreover, the proposed method has significantly better stability than several other methods.

A Study on Piecewise Polynomial Smooth Approximation to the Plus Function

In smooth support vector machine (SSVM), the plus function must be approximated by some smooth function, and the approximate error will affect the classification ability. This paper studies the smooth approximation to the plus function by piecewise polynomials. First, some standard piecewise polynomial smooth approximation problems are formulated. Then, the existence and uniqueness of solution for these problems are proved and the analytic solutions are achieved. The comparison between the results in this paper and the previous ones shows that the piecewise polynomial functions in this paper achieve better approximation to the plus function

A new pruning method for decision tree based on structural risk of leaf node

Pruning is an effective technique in improving the generalization performance of decision tree. However, most of the existing methods are time-consuming or unsuitable for small dataset. In this paper, a new pruning algorithm based on structural risk of leaf node is proposed. The structural risk is measured by the product of the accuracy and the volume (PAV) in leaf node. The comparison experiments with Cost-Complexity Pruning using cross-validation (CCP-CV) algorithm on some benchmark datasets show that PAV pruning largely reduces the time cost of CCP-CV, while the test accuracy of PAV pruning is close to that of CCP-CV.

A joint residual network with paired ReLUs activation for image super-resolution

Abstract Recently, single image super-resolution (SR) models based on deep convolutional neural network (CNN) have achieved significant advances in accuracy and speed. However, these models are not efficient enough for the image SR task. Firstly, we find that generic deep CNNs learn the low frequency features in all layers, which is redundant and leads to a slow training speed. Secondly, rectified linear unit (ReLU) only activate the positive response of the neuron, while the negative response is also worth being activated. In this paper, we propose a novel joint residual network (JRN) with three subnetworks, in which two shallow subnetworks aim to learn the low frequency information and one deep subnetwork aims to learn the high frequency information. In order to activate the negative part of the neurons and to preserve the sparsity of activation function, we propose a paired ReLUs activation scheme: one of the ReLUs is for positive activation and the other is for negative activation. The above two innovations lead to a much faster training, as well as a more efficient local structure. The proposed JRN achieves the same accuracy of a generic CNN with only 10.5% training iterations. The experiments on a wide range of images show that JRN is superior to the state-of-the-art methods both in accuracy and computational efficiency.

Improving the integration of piece wise linear representation and weighted support vector machine for stock trading signal prediction

Display Omitted The prediction problem is simplified to a weighted two-class problem.The identification of buying or selling signal from TP is done by prior information.A delay-one-day strategy is proposed to correct the predicted trading signals.A procedure for automatically selecting the threshold of PLR is provided.The curve of the average profit is steadily upward with accepted retracements. The integrated piecewise linear representation (PLR) and weighted support vector machine (PLR-WSVM) has shown success in the prediction of stock trading signals. Meanwhile drawbacks of PLR-WSVM exist particularly in a real world setting. For example, the profitability using PLR-WSVM is unstable, it is not reasonable to specify same threshold value for all stocks in PLR, and critical errors in trading signals may significantly reduce the profit. In this paper, we conduct a set of improvements to PLR-WSVM. First, most of absolute technical indicators in input variables are substituted with relative indicators since the relative indicators are generally more helpful in predicting trading signals. Second, a four-class prediction problem is converted into a two-class problem in which one class is a turning point (TP) and the other is an ordinary point. And prior domain knowledge is exploited to identify either buying or selling signals from TPs. Thirdly, a delay-one-day strategy (DODS) is proposed to further correct the predicted trading signals. DODS reduces the critical errors occurring to PLR-WSVM. Finally, a procedure for selecting a threshold in PLR is provided. The threshold is automatically selected by a given percentage of TPs in a training set. The percentage of TPs is easier to understand by investor compared with the threshold. We conduct experimental study over 20 stocks, and the results confirm the expected performance of the improved PLR-WSVM. More importantly, the improved PLR-WSVM provides steady profits in average over the stocks of interest with accepted retracements.

Training restricted Boltzmann Machine with dynamic learning rate

Restricted Boltzmann Machine (RBM) has been successfully applied to many different machine learning and pattern recognition problems. Usually, fixed learning rate (FLR) is used for training RBM. However, the reconstruction error (RCERR) with FLR may not be declined each iteration, which will result in a slow convergence speed. In this paper, we propose a method to dynamically choose the learning rate by reducing RCERR properly. The experiments on MNIST database and Caltech 101 Silhouettes database show the RBMs trained with dynamic learning rate (DLR) are better than that trained with FLR in classification accuracy and stability. It indicates DLR may be more suitable for training RBM.