Ying Tai

Nuclear Norm Based Matrix Regression with Applications to Face Recognition with Occlusion and Illumination Changes

Recently, regression analysis has become a popular tool for face recognition. Most existing regression methods use the one-dimensional, pixel-based error model, which characterizes the representation error individually, pixel by pixel, and thus neglects the two-dimensional structure of the error image. We observe that occlusion and illumination changes generally lead, approximately, to a low-rank error image. In order to make use of this low-rank structural information, this paper presents a two-dimensional image-matrix-based error model, namely, nuclear norm based matrix regression (NMR), for face representation and classification. NMR uses the minimal nuclear norm of representation error image as a criterion, and the alternating direction method of multipliers (ADMM) to calculate the regression coefficients. We further develop a fast ADMM algorithm to solve the approximate NMR model and show it has a quadratic rate of convergence. We experiment using five popular face image databases: the Extended Yale B, AR, EURECOM, Multi-PIE and FRGC. Experimental results demonstrate the performance advantage of NMR over the state-of-the-art regression-based methods for face recognition in the presence of occlusion and illumination variations.

Image Super-Resolution via Deep Recursive Residual Network

Recently, Convolutional Neural Network (CNN) based models have achieved great success in Single Image Super-Resolution (SISR). Owing to the strength of deep networks, these CNN models learn an effective nonlinear mapping from the low-resolution input image to the high-resolution target image, at the cost of requiring enormous parameters. This paper proposes a very deep CNN model (up to 52 convolutional layers) named Deep Recursive Residual Network (DRRN) that strives for deep yet concise networks. Specifically, residual learning is adopted, both in global and local manners, to mitigate the difficulty of training very deep networks, recursive learning is used to control the model parameters while increasing the depth. Extensive benchmark evaluation shows that DRRN significantly outperforms state of the art in SISR, while utilizing far fewer parameters. Code is available at https://github.com/tyshiwo/DRRN_CVPR17.

Double Nuclear Norm-Based Matrix Decomposition for Occluded Image Recovery and Background Modeling

Robust principal component analysis (RPCA) is a new emerging method for exact recovery of corrupted low-rank matrices. It assumes that the real data matrix has low rank and the error matrix is sparse. This paper presents a method called double nuclear norm-based matrix decomposition (DNMD) for dealing with the image data corrupted by continuous occlusion. The method uses a unified low-rank assumption to characterize the real image data and continuous occlusion. Specifically, we assume all image vectors form a low-rank matrix, and each occlusion-induced error image is a low-rank matrix as well. Compared with RPCA, the low-rank assumption of DNMD is more intuitive for describing occlusion. Moreover, DNMD is solved by alternating direction method of multipliers. Our algorithm involves only one operator: the singular value shrinkage operator. DNMD, as a transductive method, is further extended into inductive DNMD (IDNMD). Both DNMD and IDNMD use nuclear norm for measuring the continuous occlusion-induced error, while many previous methods use L1, L2, or other M-estimators. Extensive experiments on removing occlusion from face images and background modeling from surveillance videos demonstrate the effectiveness of the proposed methods.

Face Recognition With Pose Variations and Misalignment via Orthogonal Procrustes Regression

A linear regression-based method is a hot topic in face recognition community. Recently, sparse representation and collaborative representation-based classifiers for face recognition have been proposed and attracted great attention. However, most of the existing regression analysis-based methods are sensitive to pose variations. In this paper, we introduce the orthogonal Procrustes problem (OPP) as a model to handle pose variations existed in 2D face images. OPP seeks an optimal linear transformation between two images with different poses so as to make the transformed image best fits the other one. We integrate OPP into the regression model and propose the orthogonal Procrustes regression (OPR) model. To address the problem that the linear transformation is not suitable for handling highly non-linear pose variation, we further adopt a progressive strategy and propose the stacked OPR. As a practical framework, OPR can handle face alignment, pose correction, and face representation simultaneously. We optimize the proposed model via an efficient alternating iterative algorithm, and experimental results on three popular face databases, such as CMU PIE database, CMU Multi-PIE database, and LFW database, demonstrate the effectiveness of our proposed method.

Nuclear-L1 norm joint regression for face reconstruction and recognition with mixed noise

Abstract Occlusion, real disguise and illumination are still the common difficulties encountered in face recognition. The sparse representation based classifier (SRC) has shown a great potential in handling pixel-level sparse noise, while the nuclear norm based matrix regression (NMR) model has been demonstrated to be powerful for dealing with the image-wise structural noise. Both methods, however, might be not very effective for handling the mixed noise: the structural noise plus the sparse noise. In this paper, we present two nuclear-L 1 norm joint matrix regression (NL 1 R) models for face recognition with mixed noise, which are derived by using MAP (maximum a posteriori probability estimation). The first model considers the mixed noise as a whole, while the second model assumes the mixed noise is an additive combination of two independent componenral nts: sparse noise and structuoise. The proposed models can be solved by the alternating direction method of multipliers (ADMM). We validate the effectiveness of the proposed models through a series of experiments on face reconstruction and recognition.

Learning discriminative singular value decomposition representation for face recognition

Abstract Face representation is a critical step in face recognition. Recently, singular value decomposition (SVD) based representation methods have attracted researchers׳ attentions for their power of alleviating the facial variations. The SVD representation reveals that the SVD basis set is important for the recognition purpose and the corresponding singular values (SVs) are regulated to form a more effective representation image. However, there exists a common problem in the existing SVD based representation methods: they all empirically make a rule to regulate the SVs, which is obviously not optimal in theory. To address this problem, in this paper, we propose a novel method named learning discriminative singular value decomposition representation (LDSVDR) for face recognition. We build an individual SVD basis set for each image and then learn a common set of SVs by taking account of the information in the basis sets according to a discriminant criterion across the training images. The proposed model is solved by sequential quadratic programming (SQP) method. Extensive experiments are conducted on three popular face databases and the results demonstrate the effectiveness of our method when dealing with variations of illumination, occlusion, disguise and face sketch recognition task.

Adaptive noise dictionary construction via IRRPCA for face recognition

Recently, regression analysis has become a popular method for face recognition. Various robust regression methods have been proposed to handle with different recognition tasks. In this paper, we attempt to achieve this goal by the strategy of adding an adaptive noise dictionary (AND) to the training samples. In contrast to the previous methods, the noise dictionary (ND) is adaptive to different kinds of noise and extracted automatically. To get an effective noise dictionary, the Iteratively Reweighted Robust Principal Component Analysis (IRRPCA) is proposed. A corresponding classifier based on linear regression is presented for recognition. As this adaptive noise dictionary can describe the noise distribution of testing samples, it is robust to various kinds of noise and applicable for recognition tasks with occluded or corrupted images. This method is also extended to deal with misaligned images. Experiments are conducted on AR, Yale B, CMU PIE, CMU Multi-Pie, LFW and Pubfig databases to verify the robustness of our method to variations in occlusion, corruption, illumination, misalignment, etc. A novel noise dictionary for regression is developed.The noise dictionary is adaptive to different kinds of noise.Iteratively Reweighted Robust Principal Component Analysis is developed.Augmented Lagrangian Multiplier method is used to solve our model.An extended version is provided to deal with misaligned images.

Face Recognition with Image Misalignment via Structure Constraint Coding

Face recognition (FR) via sparse representation has been widely studied in the past several years. Recently many sparse representation based face recognition methods with simultaneous misalignment were proposed and showed interesting results. In this paper, we present a novel method called structure constraint coding (SCC) for face recognition with image misalignment. Unlike those sparse representation based methods, our method does image alignment and image representation via structure constraint based regression simultaneously. Here, we use the nuclear norm as a structure constraint criterion to characterize the error image. Compared with the sparse representation based methods, SCC is more robust for dealing with illumination variations and structural noise (especially block occlusion). Experimental results on public face databases verify the effectiveness of our method.

Exploring deep gradient information for biometric image feature representation

Abstract This paper presents a novel but simple biometric image feature representation method, called exploring deep gradient information (DGI). DGI first captures the local structure of an image by computing the histogram of gradient orientation of each macro-pixel (local patch around the reference pixel). Thus, one image can be decomposed into L sub-images (sub-orientation images) according to the gradient information of each macro-pixel since there are L bins in the local histogram. To enrich the gradient information, we also consider the gradient orientation and magnitude of original image as sub-images. For each sub-image, histogram of oriented gradient (HOG) is used to further explore the gradient orientation information. All HOG features are concatenated into one augmented super-vector. Finally, fisher linear discriminate analysis (FLDA) is applied to obtain the low-dimensional and discriminative feature vector. We evaluated the proposed method on the real-world face image datasets NUST-RWFR, Pubfig and LFW, the PolyU Finger-Knuckle-Print database and the PolyU Palmprint database. Experimental results clearly demonstrate the effectiveness of the proposed DGI compared with state-of-the-art algorithms, e.g., SIFT, HOG, LBP, POEM, LARK and IDLS.

Robust Nuclear Norm-Based Matrix Regression With Applications to Robust Face Recognition

Face recognition (FR) via regression analysis-based classification has been widely studied in the past several years. Most existing regression analysis methods characterize the pixelwise representation error via