Yinjie Lei

A Two-Phase Weighted Collaborative Representation for 3D partial face recognition with single sample

3D face recognition with the availability of only partial data (missing parts, occlusions and data corruptions) and single training sample is a highly challenging task. This paper presents an efficient 3D face recognition approach to address this challenge. We represent a facial scan with a set of local Keypoint-based Multiple Triangle Statistics (KMTS), which is robust to partial facial data, large facial expressions and pose variations. To address the single sample problem, we then propose a Two-Phase Weighted Collaborative Representation Classification (TPWCRC) framework. A class-based probability estimation is first calculated based on the extracted local descriptors as a prior knowledge. The resulting class-based probability estimation is then incorporated into the proposed classification framework as a locality constraint to further enhance its discriminating power. Experimental results on six challenging 3D facial datasets show that the proposed KMTS-TPWCRC framework achieves promising results for human face recognition with missing parts, occlusions, data corruptions, expressions and pose variations. HighlightsNovel Keypoint-based Multiple Triangle Statistics (KMTS) are proposed for 3D face representation.The proposed local descriptor is robust to partial facial data and expression/pose variations.A Two-Phase Weighted Collaborative Representation Classification (TPWCRC) framework is used to perform face recognition.The proposed classification framework can effectively address the single sample problem.State-of-the-art performance on six challenging datasets with high efficiency is achieved.

EI3D: Expression-invariant 3D face recognition based on feature and shape matching

This paper presents a local feature based shape matching algorithm for expression-invariant 3D face recognition. Each 3D face is first automatically detected from a raw 3D data and normalized to achieve pose invariance. The 3D face is then represented by a set of keypoints and their associated local feature descriptors to achieve robustness to expression variations. During face recognition, a probe face is compared against each gallery face using both local feature matching and 3D point cloud registration. The number of feature matches, the average distance of matched features, and the number of closest point pairs after registration are used to measure the similarity between two 3D faces. These similarity metrics are then fused to obtain the final results. The proposed algorithm has been tested on the FRGC v2 benchmark and a high recognition performance has been achieved. It obtained the state-of-the-art results by achieving an overall rank-1 identification rate of 97.0% and an average verification rate of 99.01% at 0.001 false acceptance rate for all faces with neutral and non-neutral expressions. Further, the robustness of our algorithm under different occlusions has been demonstrated on the Bosphorus dataset.

Recognizing Objects in 3D Point Clouds with Multi-Scale Local Features

Recognizing 3D objects from point clouds in the presence of significant clutter and occlusion is a highly challenging task. In this paper, we present a coarse-to-fine 3D object recognition algorithm. During the phase of offline training, each model is represented with a set of multi-scale local surface features. During the phase of online recognition, a set of keypoints are first detected from each scene. The local surfaces around these keypoints are further encoded with multi-scale feature descriptors. These scene features are then matched against all model features to generate recognition hypotheses, which include model hypotheses and pose hypotheses. Finally, these hypotheses are verified to produce recognition results. The proposed algorithm was tested on two standard datasets, with rigorous comparisons to the state-of-the-art algorithms. Experimental results show that our algorithm was fully automatic and highly effective. It was also very robust to occlusion and clutter. It achieved the best recognition performance on all of these datasets, showing its superiority compared to existing algorithms.

Integrating appearance features and soft biometrics for person re-identification

Matching people in different camera views, commonly referred to as person re-identification, is an inherently challenging task due to the appearance disparity caused by view change.Other factors such as low image resolution and occlusion further compound this problem. As a highly demanded technique, person re-identification has been actively studied in recent years. Most of the existing approaches either focus on feature design or distance metric learning, based on appearance features. However, due to the view change, the appearance features may significantly vary for the same subject, resulting in matching difficulties. Instead of using features from a single modality, i.e, appearance, we propose to use multimodal features to improve the re-identification accuracy. Specifically, in this work, we leverage both appearance features and soft biometrics, i.e, human characteristics such as gender, to match individuals across cameras. We build multiple graphs, each of which represent one feature modality, and the graphs are then combined and optimized to derive the similarities between a probe and the gallery subjects. The proposed method is evaluated on the VIPeR dataset with annotated soft biometric labels. The results suggest that using multimodal features, e.g, appearance and soft biometrics, can improve the matching accuracy as compared to using appearance features only, and superior performance is obtained as compared to other state-of-the-art approaches.

An HMM-SVM-based automatic image annotation approach

This paper presents a novel approach to Automatic Image Annotation (AIA) which combines both Hidden Markov Model (HMM) and Support Vector Machine (SVM). Typical image annotation methods directly map low-level features to high-level concepts and overlook the importance to mining the contextual information among the annotated keywords. The proposed HMM-SVM based approach comprises two different kinds of HMMs based on image color and texture features as the first-stage mapping scheme and an SVM which is based on the prediction results from the two HMMs as a so-called high-level classifier for final keywording. Our proposed approach assigns 1-5 keywords to each testing image. Using the Corel image dataset, Our experiments have shown that the combination of a discriminative classification and a generative model is beneficial in image annotation.

A dense flow-based framework for real-time object registration under compound motion

Abstract A moving object often has elastic and deformable surfaces (e.g., a human head). Tracking and measuring surface deformation while the object itself is also moving is a challenging, yet important problem in many video analysis tasks. For example, video-based facial expression recognition requires tracking non-rigid motions of facial features without being affected by any rigid motions of the head. In this paper, we present a generic video alignment framework to extract and characterize surface deformations accompanied by rigid-body motions with respect to a fixed reference (a canonical form). We propose a generic model for object alignment in a Bayesian framework, and rigorously show that a special case of the model results in a SIFT flow and optical flow based least-square problem. We demonstrate that dynamic programming can be used to speed up the computation of our algorithm. The proposed algorithm is evaluated on three applications, including the analysis of subtle facial muscle dynamics in spontaneous expressions, face image super-resolution, and generic object registration. Experimental results, in terms of both qualitative and quantitative measures, demonstrate the efficacy of the proposed algorithm, which can be executed in real time.

Efficient rotation estimation for 3D registration and global localization in structured point clouds

Abstract Fully automatic 3D point cloud registration for structured scenes is a highly challenging task. In this paper, an efficient rotation estimation algorithm is proposed for point clouds of structured scenes. This algorithm fully employs the geometric information of structured environment. For rotation estimation, a direction angle is defined for a point cloud and then the rotation matrix is obtained by comparing the difference between the distributions of angles. The proposed rotation estimation algorithm is used for both 3D registration and global localization. To conduct a full 3D registration, the translation parameters are estimated by aligning the centers of the corresponding points while the rotation parameters are estimated by the proposed algorithm. For global localization, a translation estimation algorithm is proposed using projection information. The point clouds are projected onto the orthogonal plane and template matching is performed on the projection images to calculate the translation vector. Experiments have been conducted on two datasets. Experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art approaches in terms of both accuracy, computational efficiency and robustness.

Boosting Multi-view Convolutional Neural Networks for 3D Object Recognition via View Saliency

2D views of objects play an important role in 3D object recognition. In this paper, we focus on 3D object recognition using the 2D projective views. The discriminativeness of each view of an object is first investigated with view saliency using 2D Zernike Moments. The proposed view saliency is then used to boost a multi-view convolutional neural network for 3D object recognition. The proposed method is compared with several state-of-the-art methods on the ModelNet dataset. Experimental results have shown that the performance of our method has been significantly improved over the existing multi-view based 3D object recognition methods.

A structured template based 3D face recognition approach

There are many challenges to achieve 2D face recognition including illumination, expression, and pose variations. However, the human face provides not only 2D texture but also rich 3D shape information. In this work, we present a novel 3D face recognition approach based on a new proposed concept termed structured template in analogy with the structured light approach. Our approach excludes the non-rigid facial region which is most affected by facial expressions. We first apply the structured template on the facial range image to extract 20 levels of stripes and convert them to pointclouds. Then we can represent a 3D facial scan by 20 levels of 3D open curves. As a result we can match the shape of two facial scans by matching the shape of their corresponding open curves. An open curve analysis algorithm is applied to calculate the geodesic distance between a pair of open curves extracted from different faces. The geodesic distance is then used as a similarity measure and two facial scans can be matched using the sum of all levels of their corresponding geodesic distance. Experiments are performed on the FRGC v2.0 dataset which demonstrate excellent performance.

An efficient 3D partial face recognition approach with single sample

3D partial face recognition under missing parts, occlusions and data corruptions is a major challenge for the practical application of the techniques of 3D face recognition. Moreover, one individual can only provide one sample for training in most practical scenarios, and thus the face recognition with single sample problem is another highly challenging task. We propose an efficient framework for 3D partial face recognition with single sample addressing both of the two problems. First, we represent a facial scan with a set of keypoint based local geometrical descriptors, which gains sufficient robustness to partial facial data along with expression/pose variations. Then, a two-step modified collaborative representation classification scheme is proposed to address the single sample recognition problem. A class-based probability estimation is given during the first classification step, and the obtained result is then incorporated into the modified collaborative representation classification as a locality constraint to improve its classification performance. Extensive experiments on the Bosphorus and FRGC v2.0 datasets demonstrate the efficiency of the proposed approach when addressing the problem of 3D partial face recognition with single sample.