Haitao Gan

Using clustering analysis to improve semi-supervised classification

Semi-supervised classification has become an active topic recently and a number of algorithms, such as Self-training, have been proposed to improve the performance of supervised classification using unlabeled data. In this paper, we propose a semi-supervised learning framework which combines clustering and classification. Our motivation is that clustering analysis is a powerful knowledge-discovery tool and it may reveal the underlying data space structure from unlabeled data. In our framework, semi-supervised clustering is integrated into Self-training classification to help train a better classifier. In particular, the semi-supervised fuzzy c-means algorithm and support vector machines are used for clustering and classification, respectively. Experimental results on artificial and real datasets demonstrate the advantages of the proposed framework.

Self-training-based face recognition using semi-supervised linear discriminant analysis and affinity propagation

Face recognition is one of the most important applications of machine learning and computer vision. The traditional supervised learning methods require a large amount of labeled face images to achieve good performance. In practice, however, labeled images are usually scarce while unlabeled ones may be abundant. In this paper, we introduce a semi-supervised face recognition method, in which semi-supervised linear discriminant analysis (SDA) and affinity propagation (AP) are integrated into a self-training framework. In particular, SDA is employed to compute the face subspace using both labeled and unlabeled images, and AP is used to identify the exemplars of different face classes in the subspace. The unlabeled data can then be classified according to the exemplars and the newly labeled data with the highest confidence are added to the labeled data, and the whole procedure iterates until convergence. A series of experiments on four face datasets are carried out to evaluate the performance of our algorithm. Experimental results illustrate that our algorithm outperforms the other unsupervised, semi-supervised, and supervised methods.

Improving motor imagery practice with synchronous action observation in stroke patients.

Background: Action observation (AO) has the potential to improve motor imagery (MI) practice in stroke patients. However, currently only a few results are available on how to use AO effectively. Objective: The aim of this study is to investigate whether MI practice can be improved more effectively by synchronous AO than by asynchronous AO. Methods: Ten patients with upper limb motor dysfunction following stroke were selected as the participants. They were divided into two groups to perform MI practice combined with a daily conventional rehabilitation for four consecutive weeks. The control group was asked to perform MI guided by asynchronous AO (MIAAO), and the experimental group was asked to perform the same MI but guided by synchronous AO (MISAO). The event-related power decrease (ERD) in sensorimotor rhythms of electroencephalograph was calculated to reflect the sensorimotor cortex activation and to assess the cortex excitability during MI. Fugl-Meyer assessment (FMA) and pinch strength test (PST) were used to assess the limb motor recovery. Results: The ERD pattern of the experimental group not only had greater amplitude and longer duration, but also included more frequency components. Furthermore, the effect sizes of ERD values between the two groups continuously increased (dES > 0.8) during the course of treatment. Moreover, the FMA and PST scores achieved with MISAO were also significantly higher than those achieved with MIAAO (p < 0.05). Conclusions: Compared with MIAAO, MISAO can enhance the excitation of sensorimotor cortex more effectively and lead to a more rapid neurorehabilitation of stroke patients.

Action recognition through discovering distinctive action parts

Recent methods based on midlevel visual concepts have shown promising capabilities in the human action recognition field. Automatically discovering semantic entities such as action parts remains challenging. In this paper, we present a method of automatically discovering distinctive midlevel action parts from video for recognition of human actions. We address this problem by learning and selecting a collection of discriminative and representative action part detectors directly from video data. We initially train a large collection of candidate exemplar-linear discriminant analysis detectors from clusters obtained by clustering spatiotemporal patches in whitened space. To select the most effective detectors from the vast array of candidates, we propose novel coverage-entropy curves (CE curves) to evaluate a detectors capability of distinguishing actions. The CE curves characterize the correlation between the representative and discriminative power of detectors. In the experiments, we apply the mined part detectors as a visual vocabulary to the task of action recognition on four datasets: KTH, Olympic Sports, UCF50, and HMDB51. The experimental results demonstrate the effectiveness of the proposed method and show the state-of-the-art recognition performance.

Semi-supervised kernel minimum squared error based on manifold structure

Kernel Minimum Squared Error (KMSE) has been receiving much attention in data mining and pattern recognition in recent years. Generally speaking, training a KMSE classifier, which is a kind of supervised learning, needs sufficient labeled examples. However, there are usually a large amount of unlabeled examples and few labeled examples in real world applications. In this paper, we introduce a semi-supervised KMSE algorithm, called Laplacian regularized KMSE (LapKMSE), which explicitly exploits the manifold structure. We construct a p nearest neighbor graph to model the manifold structure of labeled and unlabeled examples. Then, LapKMSE incorporates the structure information of labeled and unlabeled examples in the objective function of KMSE by adding a Laplacian regularized term. As a result, the labels of labeled and unlabeled examples vary smoothly along the geodesics on the manifold. Experimental results on several synthetic and real-world datasets illustrate the effectiveness of our algorithm.

Manifold regularized semi-supervised Gaussian mixture model

In the last decades, Gaussian Mixture Models (GMMs) have attracted considerable interest in data mining and pattern recognition. A GMM-based clustering algorithm models a dataset with a mixture of multiple Gaussian components and estimates the model parameters using the Expectation-Maximization (EM) algorithm. Recently, a new Locally Consistent GMM (LCGMM) has been proposed to improve the clustering performance by exploiting the local manifold structure of the data using a p nearest neighbor graph. In addition to the underlying manifold structure, many other forms of prior knowledge may guide the clustering process and improve the performance. In this paper, we introduce a Semi-Supervised LCGMM (Semi-LCGMM), where the prior knowledge is provided in the form of class labels of partial data. In particular, the new Semi-LCGMM incorporates the prior knowledge into the maximum likelihood function of the original LCGMM, and the model parameters are estimated using the EM algorithm. It is worth noting that, in our algorithm, each class may be modeled by multiple Gaussian components while in the unsupervised setting each class is modeled by a single Gaussian component. Our algorithm has shown promising results in many different applications, including clustering breast cancer data, heart disease data, handwritten digit images, human face images, and image segmentation.

Locally Adaptive Shearlet Denoising Based on Bayesian MAP Estimate

A locally adaptive Bayesian estimate for image denoising is proposed by exploiting the correlation among image shear let coefficients in a sub-band. The Laplacian distribution can model a wide range of process, from heavy-tailed to less heavy-tailed processes. This paper deduces Laplacian prior distribution based the MAP estimate formula and sub-band adaptive threshold. Finally, a simulation is carried out to show the effectiveness of the new estimate. Experiment results demonstrate that compared with classical sub-band adaptive algorithms, the new denoising method has significantly increased peak signal-to-noise ratio (PSNR) and improved the quality of subjective visual effect.

Towards designing risk-based safe Laplacian Regularized Least Squares

We propose a risk-based safe Laplacian Regularized Least Squares method.Risk degree is computed by analyzing different characteristics in RLS and LapRLS.The performance of proposed algorithm is never significantly inferior to that of RLS and LapRLS.The performance of our algorithm is relatively stable with respect to the tradeoff parameter λ. Recently, Safe Semi-Supervised Learning (S3L) has become an active topic in the Semi-Supervised Learning (SSL) field. In S3L, unlabeled data that may affect the performance of SSL both positively and negatively are exploited more safely through different risk-based strategies, and such S3L methods are expected to perform at least the same as the corresponding Supervised Learning (SL) methods. While the previously proposed S3L methods considered the risk of unlabeled data, they did not explicitly model the different risk degrees of unlabeled data on the learning procedure. Hence, we propose risk-based safe Laplacian Regularized Least Squares (RsLapRLS) by analyzing the different risk degrees of unlabeled data in this paper. Our motivation is that unlabeled data may be risky in SSL and the risk degrees are different. We assign different risk degrees to unlabeled data according to the different characteristics in supervised and semi-supervised learning. Then a risk-based tradeoff term between supervised and semi-supervised learning is integrated into the objective function of SSL. The role of risk degrees is to determine the way of exploiting the unlabeled data. Unlabeled data with large risk degrees should be exploited by SL and others by SSL. In particular, we employ Regularized Least Squares (RLS) and Laplacian RLS (LapRLS) for SL and SSL, respectively. Experimental results on several UCI and benchmark datasets show that the performance of our algorithm is never significantly inferior to RLS and LapRLS. In this way, our algorithm improves the practicability of SSL.

Enhanced manifold regularization for semi-supervised classification

Manifold regularization (MR) has become one of the most widely used approaches in the semi-supervised learning field. It has shown superiority by exploiting the local manifold structure of both labeled and unlabeled data. The manifold structure is modeled by constructing a Laplacian graph and then incorporated in learning through a smoothness regularization term. Hence the labels of labeled and unlabeled data vary smoothly along the geodesics on the manifold. However, MR has ignored the discriminative ability of the labeled and unlabeled data. To address the problem, we propose an enhanced MR framework for semi-supervised classification in which the local discriminative information of the labeled and unlabeled data is explicitly exploited. To make full use of labeled data, we firstly employ a semi-supervised clustering method to discover the underlying data space structure of the whole dataset. Then we construct a local discrimination graph to model the discriminative information of labeled and unlabeled data according to the discovered intrinsic structure. Therefore, the data points that may be from different clusters, though similar on the manifold, are enforced far away from each other. Finally, the discrimination graph is incorporated into the MR framework. In particular, we utilize semi-supervised fuzzy c-means and Laplacian regularized Kernel minimum squared error for semi-supervised clustering and classification, respectively. Experimental results on several benchmark datasets and face recognition demonstrate the effectiveness of our proposed method.

Manifold Regularized Gaussian Mixture Model for Semi-supervised Clustering

Over the last few decades, Gaussian Mixture Model (GMM) has attracted considerable interest in data mining and pattern recognition. GMM can be used to cluster a bunch of data through estimating the parameters of multiple Gaussian components using Expectation-Maximization (EM). Recently, Locally Consistent GMM (LCGMM) has been proposed to improve the clustering performance of GMM by exploiting the local manifold structure modeled by a p nearest neighbor graph. In practice, various prior knowledge may be available which can be used to guide the clustering process and improve the performance. In this paper, we introduce a semi-supervised method, called Semi-supervised LCGMM (Semi-LCGMM), where prior knowledge is provided in the form of class labels of partial data. Semi-LCGMM incorporates prior knowledge into the maximum likelihood function of LCGMM and is solved by EM. It is worth noting that in our algorithm each class has multiple Gaussian components while in the unsupervised settings each class only has one Gaussian component. Experimental results on several datasets demonstrate the effectiveness of our algorithm.