Zhe Guo

Advances in computational facial attractiveness methods

Attractiveness of a face plays an important role in many social endeavors. It influences careers like digital entertainment, modeling and acting, as well as person’s career prospect, financial status, and personal relationships. Computational approaches to exploring the nature and components of face attractiveness have been proposed, and have become an emerging topic in facial analysis research. Integrating techniques from image processing, computer vision and machine learning, this subarea aims to develop computational methods to quantify and investigate the attractiveness of a face. This paper summarizes the most recent advances in four related aspects of face attractiveness: (a) facial attractiveness prediction, (b) facial attractiveness enhancement, (c) lateral facial attractiveness and (d) 3D facial attractiveness. The motivations, innovative techniques, and significant results are summarized and discussed. The open problems in these areas and directions for future work are also briefly stated.

A landmark-based data-driven approach on 2.5D facial attractiveness computation

A purely landmark-based, data-driven method to compute three kinds of geometric facial features is performed.An incremental feature selection approach to select the most discriminative subset of attractiveness-aware features is proposed.A 2.5D hybrid attractiveness computational model is developed.Experimental results demonstrate the efficacy of the computational method. Investigating the nature and components of face attractiveness from a computational view has become an emerging topic in facial analysis research. In this paper, a multi-view (frontal and profile view, 2.5D) facial attractiveness computational model is developed to explore how face geometry affects its attractiveness. A landmark-based, data-driven method is introduced to construct a huge dimension of three kinds of geometric facial measurements, including ratios, angles, and inclinations. An incremental feature selection algorithm is proposed to systematically select the most discriminative subset of geometric features, which are finally mapped to an attractiveness score through the application of support vector regression (SVR). On a dataset of 360 facial images pre-processed from BJUT-3D Face Database and an attractiveness score dataset collected from human raters, we show that the computational model performs well with low statistic error (MSE=0.4969) and good predictability (R2=0.5756).

2.5D Facial Attractiveness Computation Based on Data-Driven Geometric Ratios

Computational approaches to investigating face attractiveness have become an emerging topic in facial analysis research. Integrating techniques from image analysis, pattern recognition and machine learning, this subarea aims to explore the nature, components and impacts of facial attractiveness and to develop computational algorithms to analyze the attractiveness of a face. In this paper we develop an attractiveness computation model for both frontal and profile images (2.5D). We focus on the role of geometric ratios in the determination of facial attractivenss. Stepwise regression is used as the feature selection method to select the discriminatory variables from a huge set of data-driven ratios. Decision tree is then used to generate an automated classifier for both frontal and profile computation models. The BJUT-3D Face Database is pre-processed and tested as our experimental dataset. The low statistic errors and high correlation indicate the accuracy of our computation models.

Label Distribution-Based Facial Attractiveness Computation by Deep Residual Learning

Two key challenges lie in the facial attractiveness computation research: the lack of discriminative face representations, and the scarcity of sufficient and complete training data. Motivated by recent promising work in face recognition using deep neural networks to learn effective features, the first challenge is expected to be addressed from a deep learning point of view. A very deep residual network is utilized to enable automatic learning of hierarchical aesthetics representation. The inspiration to deal with the second challenge comes from the natural representation of the training data, where each training face can be associated with a label (score) distribution given by human raters rather than a single label (average score). This paper, therefore, recasts facial attractiveness computation as a label distribution learning problem. Integrating these two ideas, an end-to-end attractiveness learning framework is established. We also perform feature-level fusion by incorporating the low-level geometric features to further improve the computational performance. Extensive experiments are conducted on a standard benchmark, the SCUT-FBP dataset, where our approach shows significant advantages over the other state-of-the-art work.

A Method of Effective Text Extraction for Complex Video Scene

Text information contains important information for video analysis, indexing, and retrieval. Effective and efficient text extraction has been a challenging topic in recent years. Focusing on this issue, a text extraction method for complex video scene is proposed in this paper. Multiframe corner matching and heuristic rules are combined together to detect the text region candidates, which solves the issue of Harris corner filtration for complex video scene and also improves the detection accuracy using multiframe fusion. Local texture description is then used for similarity evaluation judged by SVM. Experimental results for 4 different types of 395-frame video images show the effectiveness of the proposed method compared with 5 existing text extraction methods.

A conditionally invariant mathematical morphological framework for color images

It is difficult to extend a grayscale morphological approach to color images because total vector ordering is required for color pixels. To address this issue, we developed a kind of vector ordering method based on linear transformations from RGB to other color spaces (i.e., YUV, YIQ and YCbCr) and principal component analysis (PCA). Additionally, we propose a conditionally invariant morphological framework based on the proposed vector ordering. We also define elementary multivariate morphological operators (e.g., multivariate erosion, dilation, opening and closing), and investigate their properties with a focus on duality. The proposed framework guarantees some important properties of classical mathematical morphology, such as translation-invariance, conditional increasingness, and duality. Therefore, it is easy to extend existing grayscale morphological approaches to color images in terms f the proposed multivariate morphological framework (MMF). Simulation results show the potential abilities of MMF in color image processing, such as image filtering, reconstruction, and segmentation.

Evaluations of diffusion tensor image registration based on fiber tractography

BackgroundDiffusion Tensor Magnetic Resonance Imaging (DT-MRI, also known as DTI) measures the diffusion properties of water molecules in tissues and to date is one of the main techniques that can effectively study the microstructures of the brain in vivo. Presently, evaluation of DTI registration techniques is still in an initial stage of development.Methods and resultsIn this paper, six well-known open source DTI registration algorithms: Elastic, Rigid, Affine, DTI-TK, FSL and SyN were applied on 11 subjects from an open-access dataset, among which one was randomly chosen as the template. Eight different fiber bundles of 10 subjects and the template were obtained by drawing regions of interest (ROIs) around various structures using deterministic streamline tractography. The performances of the registration algorithms were evaluated by computing the distances and intersection angles between fiber tracts, as well as the fractional anisotropy (FA) profiles along the fiber tracts. Also, the mean squared error (MSE) and the residual MSE (RMSE) of fibers originating from the registered subjects and the template were calculated to assess the registration algorithm. Twenty-seven different fiber bundles of the 10 subjects and template were obtained by drawing ROIs around various structures using probabilistic tractography. The performances of registration algorithms on this second tractography method were evaluated by computing the spatial correlation similarity of the fibers between subjects as well as between each subject and the template.ConclusionAll experimental results indicated that DTI-TK performed the best under the study conditions, and SyN ranked just behind it.

Line detection algorithm based on adaptive gradient threshold and weighted mean shift

Line detection is a classical problem in computer vision and image processing, and it is widely used as a basic method. Most of existing line detection algorithms are based on edge information, whose discontinuity limited the detection result. Meanwhile, some other algorithms only use gradient magnitudes, and neglect the function of gradient directions. In this paper, an adaptive gradient threshold and omni-direction line growing method based on line detection with weighted mean shift procedure and 2D slice sampling strategy (referred to as LSWMSAllDir) is proposed. It makes full use of the magnitudes and directions of the gradient to detect lines in the image. Experiments on synthetic data and real scene image data showed that the improve algorithm was the most accurate when compared with Progressive Probabilistic Hough Transform (PPHT), line segment detector (LSD), parameter free edge drawing (EDPF) and original line segment detection using weighted mean shift (LSWMS) algorithms.

Multi-thread block terrain dynamic scheduling based on three-dimensional array and Sudoku

With the increasing of the scale and resolution of terrains, graphic processing hardware meet the new challenges during the terrain rendering. To solve this problem, a dynamic scheduling algorithm based on the three-dimensional array and Sudoku is proposed in this paper. Mesh optimization and texture format conversion mode is utilized to reduce the terrain data size without quality reduction. Stratified block terrains can be then built corresponding to the three-dimensional array. Finally, these block terrains are loaded and unloaded dynamically based on Sudoku strategy according to the viewpoint position. Experimental results show that the efficiency of the proposed algorithm is significantly higher than six state-of-the-art algorithms. Consequently, our algorithm has the ability to load a great amount of terrain data with high performance in frame frequency, which achieves more fluid visual experience.

Adaptive Unsymmetrical Trim-Based Morphological Filter for High-Density Impulse Noise Removal

The modified decision-based unsymmetrical trimmed median filter (MDBUTMF), which is an efficient tool for restoring images corrupted with high-density impulse noise, is only effective for certain types of images. This is because the size of the selected window is fixed and some of the center pixels are replaced by a mean value of pixels in the window. To address these issues, this paper proposes an adaptive unsymmetrical trim-based morphological filter. Firstly, a strict extremum estimation approach is used, in order to decide whether the pixel to be processed belongs to a monochrome or non-monochrome area. Then, the center pixel is replaced by a median value of pixels in a window for the monochrome area. Secondly, a relaxed extremum estimation approach is used to control the size of structuring elements. Then an adaptive structuring element is obtained and the center pixel is replaced by the output of constrained morphological operators, i.e., the minimum or maximum of pixels in a trimmed structuring element. Our experimental results show that the proposed filter is more robust and practical than the MDBUTMF. Moreover, the proposed filter provides a preferable performance compared to the existing median filters and vector median filters for high-density impulse noise removal.