Yulan Guo

3D Object Recognition in Cluttered Scenes with Local Surface Features: A Survey

3D object recognition in cluttered scenes is a rapidly growing research area. Based on the used types of features, 3D object recognition methods can broadly be divided into two categories-global or local feature based methods. Intensive research has been done on local surface feature based methods as they are more robust to occlusion and clutter which are frequently present in a real-world scene. This paper presents a comprehensive survey of existing local surface feature based 3D object recognition methods. These methods generally comprise three phases: 3D keypoint detection, local surface feature description, and surface matching. This paper covers an extensive literature survey of each phase of the process. It also enlists a number of popular and contemporary databases together with their relevant attributes.

A Comprehensive Performance Evaluation of 3D Local Feature Descriptors

A number of 3D local feature descriptors have been proposed in the literature. It is however, unclear which descriptors are more appropriate for a particular application. A good descriptor should be descriptive, compact, and robust to a set of nuisances. This paper compares ten popular local feature descriptors in the contexts of 3D object recognition, 3D shape retrieval, and 3D modeling. We first evaluate the descriptiveness of these descriptors on eight popular datasets which were acquired using different techniques. We then analyze their compactness using the recall of feature matching per each float value in the descriptor. We also test the robustness of the selected descriptors with respect to support radius variations, Gaussian noise, shot noise, varying mesh resolution, distance to the mesh boundary, keypoint localization error, occlusion, clutter, and dataset size. Moreover, we present the performance results of these descriptors when combined with different 3D keypoint detection methods. We finally analyze the computational efficiency for generating each descriptor.

Median Robust Extended Local Binary Pattern for Texture Classification

Local binary patterns (LBP) are considered among the most computationally efficient high-performance texture features. However, the LBP method is very sensitive to image noise and is unable to capture macrostructure information. To best address these disadvantages, in this paper, we introduce a novel descriptor for texture classification, the median robust extended LBP (MRELBP). Different from the traditional LBP and many LBP variants, MRELBP compares regional image medians rather than raw image intensities. A multiscale LBP type descriptor is computed by efficiently comparing image medians over a novel sampling scheme, which can capture both microstructure and macrostructure texture information. A comprehensive evaluation on benchmark data sets reveals MRELBP’s high performance—robust to gray scale variations, rotation changes and noise—but at a low computational cost. MRELBP produces the best classification scores of 99.82%, 99.38%, and 99.77% on three popular Outex test suites. More importantly, MRELBP is shown to be highly robust to image noise, including Gaussian noise, Gaussian blur, salt-and-pepper noise, and random pixel corruption.

An Accurate and Robust Range Image Registration Algorithm for 3D Object Modeling

Range image registration is a fundamental research topic for 3D object modeling and recognition. In this paper, we propose an accurate and robust algorithm for pairwise and multi-view range image registration. We first extract a set of Rotational Projection Statistics (RoPS) features from a pair of range images, and perform feature matching between them. The two range images are then registered using a transformation estimation method and a variant of the Iterative Closest Point (ICP) algorithm. Based on the pairwise registration algorithm, we propose a shape growing based multi-view registration algorithm. The seed shape is initialized with a selected range image and then sequentially updated by performing pairwise registration between itself and the input range images. All input range images are iteratively registered during the shape growing process. Extensive experiments were conducted to test the performance of our algorithm. The proposed pairwise registration algorithm is accurate, and robust to small overlaps, noise and varying mesh resolutions. The proposed multi-view registration algorithm is also very accurate. Rigorous comparisons with the state-of-the-art show the superiority of our algorithm.

A novel local surface feature for 3D object recognition under clutter and occlusion

This paper presents a highly distinctive local surface feature called the TriSI feature for recognizing 3D objects in the presence of clutter and occlusion. For a feature point, we first construct a unique and repeatable Local Reference Frame (LRF) using the implicit geometrical information of neighboring triangular faces. We then generate three signatures from the three orthogonal coordinate axes of the LRF. These signatures are concatenated and then compressed into a TriSI feature. Finally, we propose an effective 3D object recognition algorithm based on hierarchical feature matching. We tested our TriSI feature on two popular datasets. Rigorous experimental results show that the TriSI feature was highly descriptive and outperformed existing algorithms under all levels of Gaussian noise, Laplacian noise, shot noise, varying mesh resolutions, occlusion, and clutter. Moreover, we tested our TriSI-based 3D object recognition algorithm on four standard datasets. The experimental results show that our algorithm achieved the best overall recognition results on these datasets.

Local binary features for texture classification

Local Binary Patterns (LBP) have emerged as one of the most prominent and widely studied local texture descriptors. Truly a large number of LBP variants has been proposed, to the point that it can become overwhelming to grasp their respective strengths and weaknesses, and there is a need for a comprehensive study regarding the prominent LBP-related strategies. New types of descriptors based on multistage convolutional networks and deep learning have also emerged. In different papers the performance comparison of the proposed methods to earlier approaches is mainly done with some well-known texture datasets, with differing classifiers and testing protocols, and often not using the best sets of parameter values and multiple scales for the comparative methods. Very important aspects such as computational complexity and effects of poor image quality are often neglected.In this paper, we provide a systematic review of current LBP variants and propose a taxonomy to more clearly group the prominent alternatives. Merits and demerits of the various LBP features and their underlying connections are also analyzed. We perform a large scale performance evaluation for texture classification, empirically assessing forty texture features including thirty two recent most promising LBP variants and eight non-LBP descriptors based on deep convolutional networks on thirteen widely-used texture datasets. The experiments are designed to measure their robustness against different classification challenges, including changes in rotation, scale, illumination, viewpoint, number of classes, different types of image degradation, and computational complexity. The best overall performance is obtained for the Median Robust Extended Local Binary Pattern (MRELBP) feature. For textures with very large appearance variations, Fisher vector pooling of deep Convolutional Neural Networks is clearly the best, but at the cost of very high computational complexity. The sensitivity to image degradations and computational complexity are among the key problems for most of the methods considered. HighlightsA taxonomy and comprehensive survey of LBP variants.Characteristics of, and connections between LBP variants are provided.A comprehensive experimental evaluation of 32 LBP methods.Comparison of 32 LBP variants with 8 deep ConvNets features.Evaluation of robustness to rotation, illumination, scale and noise changes.Comparison of computational complexity of forty variants.

A Two-Phase Weighted Collaborative Representation for 3D partial face recognition with single sample

3D face recognition with the availability of only partial data (missing parts, occlusions and data corruptions) and single training sample is a highly challenging task. This paper presents an efficient 3D face recognition approach to address this challenge. We represent a facial scan with a set of local Keypoint-based Multiple Triangle Statistics (KMTS), which is robust to partial facial data, large facial expressions and pose variations. To address the single sample problem, we then propose a Two-Phase Weighted Collaborative Representation Classification (TPWCRC) framework. A class-based probability estimation is first calculated based on the extracted local descriptors as a prior knowledge. The resulting class-based probability estimation is then incorporated into the proposed classification framework as a locality constraint to further enhance its discriminating power. Experimental results on six challenging 3D facial datasets show that the proposed KMTS-TPWCRC framework achieves promising results for human face recognition with missing parts, occlusions, data corruptions, expressions and pose variations. HighlightsNovel Keypoint-based Multiple Triangle Statistics (KMTS) are proposed for 3D face representation.The proposed local descriptor is robust to partial facial data and expression/pose variations.A Two-Phase Weighted Collaborative Representation Classification (TPWCRC) framework is used to perform face recognition.The proposed classification framework can effectively address the single sample problem.State-of-the-art performance on six challenging datasets with high efficiency is achieved.

3D free form object recognition using rotational projection statistics

Recognizing 3D objects in the presence of clutter and occlusion is a challenging task. This paper presents a 3D free form object recognition system based on a novel local surface feature descriptor. For a randomly selected feature point, a local reference frame (LRF) is defined by calculating the eigenvectors of the covariance matrix of a local surface, and a feature descriptor called rotational projection statistics (RoPS) is constructed by calculating the statistics of the point distribution on 2D planes defined from the LRF. It finally proposes a 3D object recognition algorithm based on RoPS features. Candidate models and transformation hypotheses are generated by matching the scene features against the model features in the library, these hypotheses are then tested and verified by aligning the model to the scene. Comparative experiments were performed on two publicly available datasets and an overall recognition rate of 98.8% was achieved. Experimental results show that our method is robust to noise, mesh resolution variations and occlusion.

RoPS: A local feature descriptor for 3D rigid objects based on rotational projection statistics

The proper choice of local surface feature descriptors is a key step for an accurate and robust surface matching between different range images. This paper presents a novel 3D feature descriptor for free form objects based on rotational projection statistics. A rotation invariant local reference frame for each feature point is defined by performing an eigenvalue decomposition on the covariance matrix formed by all points lying on the local surface. The feature descriptor is then constructed by rotationally projecting the neighboring 3D points onto 2D planes and by calculating low order moments and the entropy of the 2D distribution matrix on these planes. Experiments were performed on a dataset comprised of 45 scenes, and the results show that the proposed method is robust to noise and variations in mesh resolution.

An Integrated Framework for 3-D Modeling, Object Detection, and Pose Estimation From Point-Clouds

3-D modeling, object detection, and pose estimation are three of the most challenging tasks in the area of 3-D computer vision. This paper presents a novel algorithm to perform these tasks simultaneously from unordered point-clouds. Given a set of input point-clouds in the presence of clutter and occlusion, an initial model is first constructed by performing pair-wise registration between any two point-clouds. The resulting model is then updated from the remaining point-clouds using a novel model growing technique. Once the final model is reconstructed, the instances of the object are detected and the poses of its instances in the scenes are estimated. This algorithm is automatic, model free, and does not rely on any prior information about the objects in the scene. The algorithm was comprehensively tested on the University of Western Australia data set. Experimental results show that our algorithm achieved accurate modeling, detection, and pose estimation performance.