Mingbo Zhao

Motor Bearing Fault Diagnosis Using Trace Ratio Linear Discriminant Analysis

Bearings are critical components in induction motors and brushless direct current motors. Bearing failure is the most common failure mode in these motors. By implementing health monitoring and fault diagnosis of bearings, unscheduled maintenance and economic losses caused by bearing failures can be avoided. This paper introduces trace ratio linear discriminant analysis (TR-LDA) to deal with high-dimensional non-Gaussian fault data for dimension reduction and fault classification. Motor bearing data with single-point faults and generalized-roughness faults are used to validate the effectiveness of the proposed method for fault diagnosis. Comparisons with other conventional methods, such as principal component analysis, local preserving projection, canonical correction analysis, maximum margin criterion, LDA, and marginal Fisher analysis, show the superiority of TR-LDA in fault diagnosis.

Trace Ratio Optimization-Based Semi-Supervised Nonlinear Dimensionality Reduction for Marginal Manifold Visualization

Visualizing similarity data of different objects by exhibiting more separate organizations with local and multimodal characteristics preserved is important in multivariate data analysis. Laplacian Eigenmaps (LAE) and Locally Linear Embedding (LLE) aim at preserving the embeddings of all similarity pairs in the close vicinity of the reduced output space, but they are unable to identify and separate interclass neighbors. This paper considers the semi-supervised manifold learning problems. We apply the pairwise Cannot-Link and Must-Link constraints induced by the neighborhood graph to specify the types of neighboring pairs. More flexible regulation on supervised information is provided. Two novel multimodal nonlinear techniques, which we call trace ratio (TR) criterion-based semi-supervised LAE (S2LAE) and LLE (S2LLE), are then proposed for marginal manifold visualization. We also present the kernelized S2LAE and S2LLE. We verify the feasibility of S2LAE and S2LLE through extensive simulations over benchmark real-world MIT CBCL, CMU PIE, MNIST, and USPS data sets. Manifold visualizations show that S2LAE and S2LLE are able to deliver large margins between different clusters or classes with multimodal distributions preserved. Clustering evaluations show they can achieve comparable to or even better results than some widely used methods.

Trace ratio criterion based generalized discriminative learning for semi-supervised dimensionality reduction

Dealing with high-dimensional data has always been a major problem in many pattern recognition and machine learning applications. Trace ratio criterion is a criterion that can be applicable to many dimensionality reduction methods as it directly reflects Euclidean distance between data points of within or between classes. In this paper, we analyze the trace ratio problem and propose a new efficient algorithm to find the optimal solution. Based on the proposed algorithm, we are able to derive an orthogonal constrained semi-supervised learning framework. The new algorithm incorporates unlabeled data into training procedure so that it is able to preserve the discriminative structure as well as geometrical structure embedded in the original dataset. Under such a framework, many existing semi-supervised dimensionality reduction methods such as SDA, Lap-LDA, SSDR, SSMMC, can be improved using our proposed framework, which can also be used to formulate a corresponding kernel framework for handling nonlinear problems. Theoretical analysis indicates that there are certain relationships between linear and nonlinear methods. Finally, extensive simulations on synthetic dataset and real world dataset are presented to show the effectiveness of our algorithms. The results demonstrate that our proposed algorithm can achieve great superiority to other state-of-art algorithms.

Fault diagnosis of rolling element bearings via discriminative subspace learning

The trace ratio criterion based LDA method is utilized for fault diagnosis of rolling element bearings.TR-LDA is also extended to handle the nonlinear datasets confronted in real-world fault diagnosis.We evaluate the proposed method by visualizing and classifying the rolling element bearing fault data.Simulations results show the superiority of the method in fault diagnosis of rolling element bearings. Rolling element bearings play an important role in ensuring the availability of industrial machines. Unexpected bearing failures in such machines during field operation can lead to machine breakdown, which may have some pretty severe implications. To address such concern, we extend our algorithm for solving trace ratio problem in linear discriminant analysis to diagnose faulty bearings in this paper. Our algorithm is validated by comparison with other state-of art methods based on a UCI data set, and then be extended to rolling element bearing data. Through the construction of feature data set from sensor-based vibration signals of bearing, the fault diagnosis problem is solved as a pattern classification and recognition way. The two-dimensional visualization and classification accuracy of bearing data show that our algorithm is able to recognize different bearing fault categories effectively. Thus, it can be considered as a promising method for fault diagnosis.

M-Isomap: Orthogonal Constrained Marginal Isomap for Nonlinear Dimensionality Reduction

Isomap is a well-known nonlinear dimensionality reduction (DR) method, aiming at preserving geodesic distances of all similarity pairs for delivering highly nonlinear manifolds. Isomap is efficient in visualizing synthetic data sets, but it usually delivers unsatisfactory results in benchmark cases. This paper incorporates the pairwise constraints into Isomap and proposes a marginal Isomap (M-Isomap) for manifold learning. The pairwise Cannot-Link and Must-Link constraints are used to specify the types of neighborhoods. M-Isomap computes the shortest path distances over constrained neighborhood graphs and guides the nonlinear DR through separating the interclass neighbors. As a result, large margins between both interand intraclass clusters are delivered and enhanced compactness of intracluster points is achieved at the same time. The validity of M-Isomap is examined by extensive simulations over synthetic, University of California, Irvine, and benchmark real Olivetti Research Library, YALE, and CMU Pose, Illumination, and Expression databases. The data visualization and clustering power of M-Isomap are compared with those of six related DR methods. The visualization results show that M-Isomap is able to deliver more separate clusters. Clustering evaluations also demonstrate that M-Isomap delivers comparable or even better results than some state-of-the-art DR algorithms.

Graph Based Constrained Semi-Supervised Learning Framework via Label Propagation over Adaptive Neighborhood

A new graph based constrained semi-supervised learning (G-CSSL) framework is proposed. Pairwise constraints (PC) are used to specify the types (intra- or inter-class) of points with labels. Since the number of labeled data is typically small in SSL setting, the core idea of this framework is to create and enrich the PC sets using the propagated soft labels from both labeled and unlabeled data by special label propagation (SLP), and hence obtaining more supervised information for delivering enhanced performance. We also propose a Two-stage Sparse Coding, termed TSC, for achieving adaptive neighborhood for SLP. The first stage aims at correcting the possible corruptions in data and training an informative dictionary, and the second stage focuses on sparse coding. To deliver enhanced inter-class separation and intra-class compactness, we also present a mixed soft-similarity measure to evaluate the similarity/dissimilarity of constrained pairs using the sparse codes and outputted probabilistic values by SLP. Simulations on the synthetic and real datasets demonstrated the validity of our algorithms for data representation and image recognition, compared with other related state-of-the-art graph based semi-supervised techniques.

Binary- and Multi-class Group Sparse Canonical Correlation Analysis for Feature Extraction and Classification

This paper incorporates the group sparse representation into the well-known canonical correlation analysis (CCA) framework and proposes a novel discriminant feature extraction technique named group sparse canonical correlation analysis (GSCCA). GSCCA uses two sets of variables and aims at preserving the group sparse (GS) characteristics of data within each set in addition to maximize the global interset covariance. With GS weights computed prior to feature extraction, the locality, sparsity and discriminant information of data can be adaptively determined. The GS weights are obtained from an NP-hard group-sparsity promoting problem that considers all highly correlated data within a group. By defining one of the two variable sets as the class label matrix, GSCCA is effectively extended to multiclass scenarios. Then GSCCA is theoretically formulated as a least-squares problem as CCA does. Comparative analysis between this work and the related studies demonstrate that our algorithm is more general exhibiting attractive properties. The projection matrix of GSCCA is analytically solved by applying eigen-decomposition and trace ratio (TR) optimization. Extensive benchmark simulations are conducted to examine GSCCA. Results show that our approach delivers promising results, compared with other related algorithms.

Pairwise Sparsity Preserving Embedding for Unsupervised Subspace Learning and Classification

Two novel unsupervised dimensionality reduction techniques, termed sparse distance preserving embedding (SDPE) and sparse proximity preserving embedding (SPPE), are proposed for feature extraction and classification. SDPE and SPPE perform in the clean data space recovered by sparse representation and enhanced Euclidean distances over noise removed data are employed to measure pairwise similarities of points. In extracting informative features, SDPE and SPPE aim at preserving pairwise similarities between data points in addition to preserving the sparse characteristics. This paper calculates the sparsest representation of all vectors jointly by a convex optimization. The sparsest codes enable certain local information of data to be preserved, and can endow SDPE and SPPE a natural discriminating power, adaptive neighborhood and robust characteristic against noise and errors in delivering low-dimensional embeddings. We also mathematically show SDPE and SPPE can be effectively extended for discriminant learning in a supervised manner. The validity of SDPE and SPPE is examined by extensive simulations. Comparison with other related state-of-the-art unsupervised algorithms show that promising results are delivered by our techniques.

A two-dimensional Neighborhood Preserving Projection for appearance-based face recognition

This paper presents a two-dimensional Neighborhood Preserving Projection (2DNPP) for appearance-based face representation and recognition. 2DNPP enables us to directly use a feature input of 2D image matrices rather than 1D vectors. We use the same neighborhood weighting procedure that is involved in NPP to form the nearest neighbor affinity graph. Theoretical analysis of the connection between 2DNPP and other 2D methods is presented as well. We conduct extensive experimental verifications to evaluate the performance of 2DNPP on three face image datasets, i.e. ORL, UMIST, and AR face datasets. The results corroborate that 2DNPP outperforms the standard NPP approach across all experiments with respect to recognition rate and training time. 2DNPP delivers consistently promising results compared with other competing methods such as 2DLPP, 2DLDA, 2DPCA, ONPP, OLPP, LPP, LDA, and PCA.

Automatic image annotation via compact graph based semi-supervised learning

The insufficiency of labeled samples is major problem in automatic image annotation. However, unlabeled samples are readily available and abundant. Hence, semi-supervised learning methods, which utilize partly labeled samples and a large amount of unlabeled samples, have attracted increased attention in the field of image annotation. During the past decade, graph-based semi-supervised learning has been becoming one of the most important research areas in semi-supervised learning. In this paper, we propose a novel and effective graph based semi-supervised learning method for image annotation. The new method is derived by a compact graph that can well grasp the manifold structure. In addition, we theoretically prove that the proposed semi-supervised learning method can be analyzed under a regularized framework. It can also be easily extended to deal with out-of-sample data. Simulation results show that the proposed method can achieve better performance compared with other state-of-the-art graph based semi-supervised learning methods.