Jianfeng Ren

Noise-Resistant Local Binary Pattern With an Embedded Error-Correction Mechanism

Local binary pattern (LBP) is sensitive to noise. Local ternary pattern (LTP) partially solves this problem. Both LBP and LTP, however, treat the corrupted image patterns as they are. In view of this, we propose a noise-resistant LBP (NRLBP) to preserve the image local structures in presence of noise. The small pixel difference is vulnerable to noise. Thus, we encode it as an uncertain state first, and then determine its value based on the other bits of the LBP code. It is widely accepted that most of the image local structures are represented by uniform codes and noise patterns most likely fall into the non-uniform codes. Therefore, we assign the value of an uncertain bit hence as to form possible uniform codes. Thus, we develop an error-correction mechanism to recover the distorted image patterns. In addition, we find that some image patterns such as lines are not captured in uniform codes. Those line patterns may appear less frequently than uniform codes, but they represent a set of important local primitives for pattern recognition. Thus, we propose an extended noise-resistant LBP (ENRLBP) to capture line patterns. The proposed NRLBP and ENRLBP are more resistant to noise compared with LBP, LTP, and many other variants. On various applications, the proposed NRLBP and ENRLBP demonstrate superior performance to LBP/LTP variants.

Optimizing LBP Structure For Visual Recognition Using Binary Quadratic Programming

Local binary pattern (LBP) and its variants have shown promising results in visual recognition applications. However, most existing approaches rely on a pre-defined structure to extract LBP features. We argue that the optimal LBP structure should be task-dependent and propose a new method to learn discriminative LBP structures. We formulate it as a point selection problem: Given a set of point candidates, the goal is to select an optimal subset to compose the LBP structure. In view of the problems of current feature selection algorithms, we propose a novel Maximal Joint Mutual Information criterion. Then, the point selection is converted into a binary quadratic programming problem and solved efficiently via the branch and bound algorithm. The proposed LBP structures demonstrate superior performance to the state-of-the-art approaches on classifying both spatial patterns in scene recognition and spatial-temporal patterns in dynamic texture recognition.

Relaxed local ternary pattern for face recognition

Local binary pattern (LBP) is sensitive to noise. Local ternary pattern (LTP) partially solves this problem by encoding the small pixel difference into a third state. The small pixel difference may be easily overwhelmed by noise. Thus, it is difficult to precisely determine its sign and magnitude. In this paper, we propose the concept of uncertain state to encode the small pixel difference. We do not care its sign and magnitude, and encode it as both 0 and 1 with equal probability. The proposed Relaxed LTP is tested on the CMU-PIE database, the extended Yale B database and the O2FN mobile face database. Superior performance is demonstrated compared with LBP and LTP.

Learning LBP structure by maximizing the conditional mutual information

Local binary patterns of more bits extracted in a large structure have shown promising results in visual recognition applications. This results in very high-dimensional data so that it is not feasible to directly extract features from the LBP histogram, especially for a large-scale database. Instead of extracting features from the LBP histogram, we propose a new approach to learn discriminative LBP structures for a specific application. Our objective is to select an optimal subset of binarized-pixel-difference features to compose the LBP structure. As these features are strongly correlated, conventional feature-selection methods may not yield a desirable performance. Thus, we propose an incremental Maximal-Conditional-Mutual-Information scheme for LBP structure learning. The proposed approach has demonstrated a superior performance over the state-of-the-arts results on classifying both spatial patterns such as texture classification, scene recognition and face recognition, and spatial-temporal patterns such as dynamic texture recognition. HighlightsWe propose a new approach to tackle high-dimensional LBP features.It discovers optimal LBP structure to generate discriminative features.We propose a MCMI scheme for LBP structure learning to handle pixel correlation.It demonstrates a superior performance to SOTA on various visual applications.

Dynamic texture recognition using enhanced LBP features

This paper addresses the challenge of recognizing dynamic textures based on spatial-temporal descriptors. Dynamic textures are composed of both spatial and temporal features. The histogram of local binary pattern (LBP) has been used in dynamic texture recognition. However, its performance is limited by the reliability issues of the LBP histograms. In this paper, two learning-based approaches are proposed to remove the unreliable information in LBP features by utilizing Principal Histogram Analysis. Furthermore, a super histogram is proposed to improve the reliability of the LBP histograms. The temporal information is partially transferred to the super histogram. The proposed approaches are evaluated on two widely used benchmark databases: UCLA and Dyntex++ databases. Superior performance is demonstrated compared with the state of the arts.

Eye detection based on rank order filter

A novel eye detection algorithm based on rank order filter is proposed in this paper. Features such as eyeball is round and darker than surrounding pixels are widely used in eye detection. However quite often eyeball is distorted by iris reflection or other obstacles around eyeball. Rank order filter pair is designed to tackle these problems. One rank order filter is applied on central pixels to emphasize the darkness of the eyeball pixels and another rank order filter is applied on surrounding pixels to emphasize the brightness of the surrounding pixels. Then the difference of those two filter outputs is an important clue for eye detection, since regions near eyeball will yield large response. Those pixels of large response are selected as eyeball candidates, grouped and further verified by a series of geometric constraints. It is tested for 4095 face images with large variations in illuminations, hair styles, facial expressions and partial occlusions and detection rate as high as 98.97% is achieved.

Modelling multi-party interactions among virtual characters, robots, and humans

3D virtual humans and physical human-like robots can be used to interact with people in a remote location in order to increase the feeling of presence. In a telepresence setup, their behaviors are driven by real participants. We envision that in the absence of the real users, when they have to leave or they do not want to do a repetitive task, the control of the robots can be handed to an artificial intelligence component to sustain the ongoing interaction. At the point when human-mediated interaction is required again, control can be returned to the real users. One of the main challenges in telepresence research is the adaptation of 3D position and orientation of the remote participants to the actual physical environment to have appropriate eye contact and gesture awareness in a group conversation. In case the human behind the robot and/or virtual human leaves, multi-party interaction should be handed to an artificial intelligence component. In this paper, we discuss the challenges in autonomous multi-party interaction among virtual characters, human-like robots, and real participants, and describe a prototype system to study these challenges.

Learning binarized pixel-difference pattern for scene recognition

Local binary pattern (LBP) and its variants have been used in scene recognition. However, most existing approaches rely on a pre-defined LBP structure to extract features. Those pre-defined structures can be generalized as the patterns constructed from the binarized pixel differences in a local neighborhood. Instead of using a handcraft structure, we propose to learn binarized pixel-difference patterns (BPP). We cast the problem as a feature selection problem and solve it by an incremental search via the criterion of minimum-redundancy-maximum-relevance. Then, BPP features are extracted based on the structures derived. On two challenging scene recognition databases, the proposed approach significantly outperforms the state of the arts.

Fast eye localization based on pixel differences

A novel fast eye localization algorithm based on pixel differences is presented, which is suitable for face recognition system on mobile device. It is based on the fact that eyeball is dark and round. A binary eye map is obtained by choosing those pixels darker than surrounding; then it is filtered by a rank order filter; connected regions in the eye map are then labeled by their geometric centers; best suitable eyeball pair is selected based on a set of geometric constraints. If no eyeball pair is detected, the algorithm is repeated iteratively until one pair is found. The algorithm is fast since it converts the gray level image to a binary eye map at the beginning. The algorithm is tested on our own face database, which consists of 4095 images of size 250×200. Detection rate is 93.04% when the tolerance is 0.7 times of eyeball width.

A chi-squared-transformed subspace of LBP histogram for visual recognition.

Local binary pattern (LBP) and its variants have been widely used in many recognition tasks. Subspace approaches are often applied to the LBP feature in order to remove unreliable dimensions, or to derive a compact feature representation. It is well-known that subspace approaches utilizing up to the second-order statistics are optimal only when the underlying distribution is Gaussian. However, due to its nonnegative and simplex constraints, the LBP feature deviates significantly from Gaussian distribution. To alleviate this problem, we propose a chi-squared transformation (CST) to transfer the LBP feature to a feature that fits better to Gaussian distribution. The proposed CST leads to the formulation of a two-class classification problem. Due to its asymmetric nature, we apply asymmetric principal component analysis (APCA) to better remove the unreliable dimensions in the CST feature space. The proposed CST-APCA is evaluated extensively on spatial LBP for face recognition, protein cellular classification, and spatial-temporal LBP for dynamic texture recognition. All experiments show that the proposed feature transformation significantly enhances the recognition accuracy.