Shu Hung Leung

Segmentation of color lip images by spatial fuzzy clustering

In this paper, we describe the application of a novel spatial fuzzy clustering algorithm to the lip segmentation problem. The proposed spatial fuzzy clustering algorithm is able to take into account both the distributions of data in feature space and the spatial interactions between neighboring pixels during clustering. By appropriate pre- and postprocessing utilizing the color and shape properties of the lip region, successful segmentation of most lip images is possible. Comparative study with some existing lip segmentation algorithms such as the hue filtering algorithm and the fuzzy entropy histogram thresholding algorithm has demonstrated the superior performance of our method.

Gradient-based variable forgetting factor RLS algorithm in time-varying environments

In this paper, a new control mechanism for the variable forgetting factor (VFF) of the recursive least square (RLS) adaptive algorithm is presented. The control algorithm is basically a gradient-based method of which the gradient is derived from an improved mean square error analysis of RLS. The new mean square error analysis exploits the correlation of the inverse of the correlation matrix with itself that yields improved theoretical results, especially in the transient and steady-state mean square error. It is shown that the theoretical analysis is close to simulation results for different forgetting factors and different model orders. The analysis yields a dynamic equation of mean square error that can be used to derive a dynamic equation of the gradient of mean square error to control the forgetting factor. The dynamic equation can produce a positive gradient when the error is large and a negative gradient when the error is in the steady state. Compared with other variable forgetting factor algorithms, the new control algorithm gives fast tracking and small mean square model error for different signal-to-noise ratios (SNRs).

Lip image segmentation using fuzzy clustering incorporating an elliptic shape function

Recently, lip image analysis has received much attention because its visual information is shown to provide improvement for speech recognition and speaker authentication. Lip image segmentation plays an important role in lip image analysis. In this paper, a new fuzzy clustering method for lip image segmentation is presented. This clustering method takes both the color information and the spatial distance into account while most of the current clustering methods only deal with the former. In this method, a new dissimilarity measure, which integrates the color dissimilarity and the spatial distance in terms of an elliptic shape function, is introduced. Because of the presence of the elliptic shape function, the new measure is able to differentiate the pixels having similar color information but located in different regions. A new iterative algorithm for the determination of the membership and centroid for each class is derived, which is shown to provide good differentiation between the lip region and the nonlip region. Experimental results show that the new algorithm yields better membership distribution and lip shape than the standard fuzzy c-means algorithm and four other methods investigated in the paper.

Magnified gradient function with deterministic weight modification in adaptive learning

This work presents two novel approaches, backpropagation (BP) with magnified gradient function (MGFPROP) and deterministic weight modification (DWM), to speed up the convergence rate and improve the global convergence capability of the standard BP learning algorithm. The purpose of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function, while the main objective of DWM is to reduce the system error by changing the weights of a multilayered feedforward neural network in a deterministic way. Simulation results show that the performance of the above two approaches is better than BP and other modified BP algorithms for a number of learning problems. Moreover, the integration of the above two approaches forming a new algorithm called MDPROP, can further improve the performance of MGFPROP and DWM. From our simulation results, the MDPROP algorithm always outperforms BP and other modified BP algorithms in terms of convergence rate and global convergence capability.

Lip contour extraction from color images using a deformable model

The use of visual information from lip movements can improve the accuracy and robustness of a speech recognition system. In this paper, a region-based lip contour extraction algorithm based on deformable model is proposed. The algorithm employs a stochastic cost function to partition a color lip image into lip and non-lip regions such that the joint probability of the two regions is maximized. Given a discrete probability map generated by spatial fuzzy clustering, we show how the optimization of the cost function can be done in the continuous setting. The region-based approach makes the algorithm more tolerant to noise and artifacts in the image. It also allows larger region of attraction, thus making the algorithm less sensitive to initial parameter settings. The algorithm works on unadorned lips and accurate extraction of lip contour is possible.

Robust lip region segmentation for lip images with complex background

Robust and accurate lip region segmentation is of vital importance for lip image analysis. However, most of the current techniques break down in the presence of mustaches and beards. With mustaches and beards, the background region becomes complex and inhomogeneous. We propose in this paper a novel multi-class, shape-guided FCM (MS-FCM) clustering algorithm to solve this problem. For this new approach, one cluster is set for the object, i.e. the lip region, and a combination of multiple clusters for the background which generally includes the skin region, lip shadow or beards. The proper number of background clusters is derived automatically which maximizes a cluster validity index. A spatial penalty term considering the spatial location information is introduced and incorporated into the objective function such that pixels having similar color but located in different regions can be differentiated. This facilitates the separation of lip and background pixels that otherwise are inseparable due to the similarity in color. Experimental results show that the proposed algorithm provides accurate lip-background partition even for the images with complex background features like mustaches and beards.

Automatic lip contour extraction from color images

Visual information from lip shapes and movements helps improve the accuracy and robustness of a speech recognition system. In this paper, a new region-based lip contour extraction algorithm that combines the merits of the point-based model and the parametric model is presented. Our algorithm uses a 16-point lip model to describe the lip contour. Given a robust probability map of the color lip image generated by the FCMS (fuzzy clustering method incorporating shape function) algorithm, a region-based cost function that maximizes the joint probability of the lip and non-lip region can be established. Then an iterative point-driven optimization procedure has been developed to fit the lip model to the probability map. In each iteration, the adjustment of the 16 lip points is governed by three pieces of quadratic curves that constrain the points to form a physical lip shape. Experiments show that the proposed approach provides satisfactory results for 5000 unadorned lip images of over 20 individuals. A real-time lip contour extraction system has also been implemented.

Gradient based variable forgetting factor RLS algorithm

An accurate new variable forgetting factor recursive least-square adaptive algorithm is derived. An improved mean square behaviour analysis is presented, which shows that the theoretical analysis and the simulation results are close to each other. The control of the forgetting factor is based on the dynamic equation of the gradient of mean square error. Compared with other variable forgetting factor algorithms, the new algorithm provides fast tracking and small mean square model error, and its performance will not be degraded much even in low signal-to-noise ratios.

A new optimization procedure for extracting the point-based lip contour using active shape model

This paper presents a new optimization procedure for extracting the point-based lip contour using the active shape model (ASM). A 14-point ASM lip model is used to describe the lip contour. With the aid of fuzzy clustering analysis, a probability map of the color lip image is obtained and a region-based cost function is established. The new optimization procedure operates on the spatial domain (actual contour points) and all the points are pulled towards their desirable locations in each iteration. Hence, the lip contour evolution becomes better controlled and consequently fast convergence is achieved. The new procedure can also achieve real-time performance on lip contour extraction and tracking from lip image sequence.

Clipping on COFDM with phase on demand

This letter presents a simple and effective peak reduction method to achieve a desired peak-to-average power ratio (PAPR). The method is called phase on demand in which a fixed phase vector is added to input vectors only if it helps reduce the peak of the orthogonal frequency-division multiplexing (OFDM) signal otherwise no phase is added. Repeated clipping and filtering is applied whenever the PAPR exceeds a desired value. To avoid sending side information about the phase, a maximum-likelihood phase detector is derived. Simulation results show that the new method is an efficient peak reduction technique for coded OFDM (COFDM).