Yun Sheng

Automatic Single View-Based 3-D Face Synthesis for Unsupervised Multimedia Applications

Various 3-D face synthesis techniques have been proposed and extensively used in many applications. Compared with others, single view-based face synthesis technology allows unsupervised 3-D face reconstruction without any offline operations. Although many algorithms have been published, automatic and robust single view-based 3-D face synthesis still remains unsolved. In contrast to other methods, the single view-based 3-D face synthesis algorithm conducted in this paper enables automated 3-D face synthesis from an arbitrary head-and-shoulder image with the complex background. The developed system first detects the face using Bayesian skin-tone classification based on only the chrominance component, Cr. Based on the detected face, a few salient facial features, such as the corners of the eyebrows and contours of the eyes, mouth, and chin are in turn extracted using variant algorithms, including a dynamic chin extraction mechanism that will be detailed in this paper. Then, face model adaptation consisting of both global and local adaptations is imposed, according to geometric information provided by the extracted facial features. Finally, the 3-D specific face is synthesized using the adapted 3-D face model with a texture map directly derived from the input face image, followed by the implementation of facial animation using this synthesized face.

Facial geometry parameterisation based on Partial Differential Equations

Geometric modelling using Partial Differential Equations (PDEs) has been gradually recognised due to its smooth instinct, as well as the ability to generate a variety of geometric shapes by intuitively manipulating a relatively small set of PDE boundary curves. In this paper we explore and demonstrate the feasibility of the PDE method in facial geometry parameterisation. The geometry of a generic face is approximated by evaluating spectral solutions to a group of fourth order elliptic PDEs. Our PDE-based parameterisation scheme can produce and animate a high-resolution 3D face with a relatively small number of parameters. By taking advantage of parametric representation, the PDE method can use one fixed animation scheme to manipulate the facial geometry in varying Levels of Detail (LODs), without any further process.

PDE-Based Facial Animation: Making the Complex Simple

Direct parameterisation is among the most widely used facial animation techniques but requires complicated ways to animate face models which have complex topology. This paper develops a simple solution by introducing a PDE-based facial animation scheme. Using a PDE face model means we only needs to animate a group of boundary curves without using any other conventional surface interpolation algorithms. We describe the basis of the method and show results from a practical implementation.

Image-based embroidery modeling and rendering

Embroidery is a traditional handicraft of sewing stitches into fabric or other materials in different patterns, and this ancient non‐photorealistic art form has not drawn enough attention thus far. In this paper, we present an image‐based method to simulate the traditional embroidery art. The method combines stroke‐based rendering techniques with the Phong lighting model to create picturesque embroidery‐like images. We first build a 3D stitch model and derive some most commonly used stitch patterns from it. Then we preprocess the input image by segmenting it into regions, from which the parameters to specify stitch patterns are obtained. Finally, we apply stitches back onto the desired regions and render them under a virtual light source. Experimental results show that our method, different from the existing schemes, is capable of performing fine embroidery simulations with the effects of lighting and shading based on an input image. Copyright

AUTOMATIC FACE SYNTHESIS AND ANALYSIS. A QUICK SURVEY

Considerable interest has been received in automatic face synthesis and analysis over the last three decades. This paper surveys the current state of the art in face synthesis, and also presents face detection and eye detection selected algorithms along with facial feature extraction approach based on using Harris corner detector in face analysis.

A heuristic convexity measure for 3D meshes

In this paper we propose a heuristic convexity measure for 3D meshes. Built upon a state-of-the-art convexity measure that employs a time-consuming genetic algorithm for optimization, our new measure projects only once a given 3D mesh onto the orthogonal 2D planes along its principal directions for an initial estimation of mesh convexity, followed by a correction calculation based on mesh slicing. Our measure experimentally shows several advantages over the state-of-the-art one: first, it accelerates the overall computation by approximately an order of magnitude; second, it properly handles those bony meshes usually overestimated by the state-of-the-art measure; third, it improves the accuracy of the state-of-the-art measure in 3D mesh retrieval.

A PDE-based head visualization method with CT data

In this paper, we extend the use of the partial differential equation (PDE) method to head visualization with computed tomography (CT) data and show how the two primary medical visualization means, surface reconstruction, and volume rendering can be integrated into one single framework through PDEs. Our scheme first performs head segmentation from CT slices using a variational approach, the output of which can be readily used for extraction of a small set of PDE boundary conditions. With the extracted boundary conditions, head surface reconstruction is then executed. Because only a few slices are used, our method can perform head surface reconstruction more efficiently in both computational time and storage cost than the widely used marching cubes algorithm. By elaborately introducing a third parameter w to the PDE method, a solid head can be created, based on which the head volume is subsequently rendered with 3D texture mapping. Instead of designing a transfer function, we associate the alpha value of texels of the 3D texture with the PDE parameter w through a linear transform. This association enables the production of a visually translucent head volume. The experimental results demonstrate the feasibility of the developed head visualization method. Copyright

Graph Cut Based Mesh Segmentation Using Feature Points and Geodesic Distance

Both prominent feature points and geodesic distance are key factors for mesh segmentation. With these two factors, this paper proposes a graph cut based mesh segmentation method. The mesh is first preprocessed by Laplacian smoothing. According to the Gaussian curvature, candidate feature points are then selected by a predefined threshold. With DBSCAN (Density-Based Spatial Clustering of Application with Noise), the selected candidate points are separated into some clusters, and the points with the maximum curvature in every cluster are regarded as the final feature points. We label these feature points, and regard the faces in the mesh as nodes for graph cut. Our energy function is constructed by utilizing the ratio between the geodesic distance and the Euclidean distance of vertex pairs of the mesh. The final segmentation result is obtained by minimizing the energy function using graph cut. The proposed algorithm is pose-invariant and can robustly segment the mesh into different parts in line with the selected feature points.

Dual-Way Guided Depth Image Inpainting with RGBD Image Pairs

Raw depth images acquired by low-cost range imaging sensors, such as Kinect v1 and Kinect v2, usually contain invalid regions without depth information and suffer from noise. Although many approaches are proposed to address the problems, the robustness of these approaches still needs enhancement. This paper introduces a dual-way guided inpainting method with RGBD image pairs to restore the missing depth values of invalid regions and to eliminate noise caused by acquisition apparatus. By leveraging the structural difference between the colour and depth images, the colour image is first segmented with watershed segmentation and then merged under the guidance of the simultaneously captured depth image, followed by inpainting the depth image guided by the merged colour image using Radial Basis Functions (RBFs). The proposed framework of the dual-way guided approach with the RBFs is new for depth image inpainting and outperforms the existing state-of-the-art approaches in the experimental evaluations.

A PDE patch-based spectral method for progressive mesh compression and mesh denoising

The development of the patchwise partial differential equation (PDE) framework a few years ago has paved the way for the PDE method to be used in mesh signal processing. In this paper, we, for the first time, extend the use of the PDE method to progressive mesh compression and mesh denoising. We, meanwhile, upgrade the existing patchwise PDE method in patch merging, mesh partitioning, and boundary extraction to accommodate mesh signal processing. In our new method, an arbitrary mesh model is partitioned into patches, each of which can be represented by a small set of coefficients of its PDE spectral solution. Since low-frequency components contribute more to the reconstructed mesh than high-frequency ones, we can achieve progressive mesh compression and mesh denoising by manipulating the frequency terms of the PDE solution. Experimental results demonstrate the feasibility of our method in both progressive mesh compression and mesh denoising.