Syed Afaq Ali Shah

Iterative deep learning for image set based face and object recognition

We present a novel technique for image set based face/object recognition, where each gallery and query example contains a face/object image set captured from different viewpoints, background, facial expressions, resolution and illumination levels. While several image set classification approaches have been proposed in recent years, most of them represent each image set as a single linear subspace, mixture of linear subspaces or Lie group of Riemannian manifold. These techniques make prior assumptions in regards to the specific category of the geometric surface on which images of the set are believed to lie. This could result in a loss of discriminative information for classification. This paper alleviates these limitations by proposing an Iterative Deep Learning Model (IDLM) that automatically and hierarchically learns discriminative representations from raw face and object images. In the proposed approach, low level translationally invariant features are learnt by the Pooled Convolutional Layer (PCL). The latter is followed by Artificial Neural Networks (ANNs) applied iteratively in a hierarchical fashion to learn a discriminative non-linear feature representation of the input image sets. The proposed technique was extensively evaluated for the task of image set based face and object recognition on YouTube Celebrities, Honda/UCSD, CMU Mobo and ETH-80 (object) dataset, respectively. Experimental results and comparisons with state-of-the-art methods show that our technique achieves the best performance on all these datasets.

3D-Div: A novel local surface descriptor for feature matching and pairwise range image registration

This paper presents a novel local surface descriptor, called 3D-Div. The proposed descriptor is based on the concept of 3D vector fields divergence, extensively used in electromagnetic theory. To generate a 3D-Div descriptor of a 3D surface, a keypoint is first extracted on the 3D surface, then a local patch of a certain size is selected around that keypoint. A Local Reference Frame (LRF) is then constructed at the keypoint using all points forming the patch. A normalized 3D vector field is then computed at each point in the patch and referenced with LRF vectors. The 3D-Div descriptors are finally generated as the divergence of the reoriented 3D vector field. We tested our proposed descriptor on the low resolution Washington RGB-D (Kinect) object dataset. Performance was evaluated for the tasks of feature matching and pairwise range image registration. Experimental results showed that the proposed 3D-Div is 88% more computationally efficient and 47% more accurate than commonly used Spin Image (SI) descriptors.

A novel 3D vorticity based approach for automatic registration of low resolution range images

This paper tackles the problem of feature matching and range image registration. Our approach is based on a novel set of discriminating three-dimensional (3D) local features, named 3D-Vor (Vorticity). In contrast to conventional local feature representation techniques, which use the vector field (i.e. surface normals) to just construct their local reference frames, the proposed feature representation exploits the vorticity of the vector field computed at each point of the local surface to capture the distinctive characteristics at each point of the underlying 3D surface. The 3D-Vor descriptors of two range images are then matched using a fully automatic feature matching algorithm which identifies correspondences between the two range images. Correspondences are verified in a local validation step of the proposed algorithm and used for the pairwise registration of the range images. Quantitative results on low resolution Kinect 3D data (Washington RGB-D dataset) show that our proposed automatic registration algorithm is accurate and computationally efficient. The performance evaluation of the proposed descriptor was also carried out on the challenging low resolution Washington RGB-D (Kinect) object dataset, for the tasks of automatic range image registration. Reported experimental results show that the proposed local surface descriptor is robust to resolution, noise and more accurate than state-of-the-art techniques. It achieves 90% registration accuracy compared to 50%, 69.2% and 52% for spin image, 3D SURF and SISI/LD-SIFT descriptors, respectively. HighlightsA novel local surface descriptor (3D-Vor) is proposed for surface representation.The proposed 3D-Vor exploits the vector field?s vorticity.A novel pairwise registration algorithm is also proposed.3D-Vor is tested on low resolution dataset for range image registration.3D-Vor based registration achieves 90% accuracy on low resolution data and outperforms state-of-the-art techniques.

A Novel Local Surface Description for Automatic 3D Object Recognition in Low Resolution Cluttered Scenes

Local surface description is a critical stage for feature matching and recognition tasks. This paper presents a rotation invariant local surface descriptor, called 3D-Div. The proposed descriptor is based on the concept of 3D vector fields divergence, extensively used in electromagnetic theory. To generate a 3D-Div descriptor of a 3D surface, a local surface patch is parameterized around a randomly selected 3D point at a fixed scale. A unique Local Reference Frame (LRF) is then constructed at that 3D point using all the neighboring points forming the patch. A normalized 3D vector field is then computed at each point in the patch and referenced with LRF vectors. The 3D-Div descriptor is finally generated as the divergence of the reoriented 3D vector field. We tested our proposed descriptor on the challenging low resolution Washington RGB-D (Kinect) object dataset, for the task of automatic 3D object recognition. Reported experimental results show that 3D-Div based recognition achieves 93% accuracy as compared to 85% for existing state-of-the-art depth kernel descriptors [2].

A novel feature representation for automatic 3D object recognition in cluttered scenes

We present a novel local surface description technique for automatic three dimensional (3D) object recognition. In the proposed approach, highly repeatable keypoints are first detected by computing the divergence of the vector field at each point of the surface. Being a differential invariant of curves and surfaces, the divergence captures significant information about the surface variations at each point. The detected keypoints are pruned to only retain the keypoints which are associated with high divergence values. A keypoint saliency measure is proposed to rank these keypoints and select the best ones. A novel integral invariant local surface descriptor, called 3D-Vor, is built around each keypoint by exploiting the vorticity of the vector field at each point of the local surface. The proposed descriptor combines the strengths of signature-based methods and integral invariants to provide robust local surface description. The performance of the proposed fully automatic 3D object recognition technique was rigorously tested on three publicly available datasets. Our proposed technique is shown to exhibit superior performance compared to state-of-the-art techniques. Our keypoint detector and descriptor based algorithm achieves recognition rates of 100%, 99.35% and 96.2% respectively, when tested on the Bologna, UWA and Ca׳ Foscari Venezia datasets.

Automatic object detection using objectness measure

Object detection is an important step towards object recognition. A robust object detection system is one that can detect an object of any class. In this paper, we present a fully automatic approach to object detection based on an objectness measure. The proposed automatic object detection approach quantifies the likelihood for an image window to encompass objects in the image. It can discriminate between multiple objects in a scene, with individual windows capturing each detected object. Most importantly, the proposed approach does not require any manual input. We tested this approach on the challenging PASCAL VOC 07 dataset. Experimental results show that our approach provides a more accurate estimation of the required number of windows for an input image. The proposed technique is computationally efficient and takes less than 4 sec. per image.

Performance Evaluation of 3D Local Surface Descriptors for Low and High Resolution Range Image Registration

Despite the advent and popularity of low-cost commercial sensors (e.g., Microsoft Kinect), research in 3D vision still primarily focuses on the development of advanced algorithms geared towards high resolution data. This paper presents a comparative performance evaluation of renowned state-of-the-art 3D local surface descriptors for the task of registration of both high and low resolution range image data. The datasets used in these experiments are the renowned high resolution Stanford 3D models dataset and challenging low resolution Washington RGB-D object dataset. Experimental results show that the performance of certain local surface descriptors is significantly affected by low resolution data.

A novel algorithm for efficient depth segmentation using low resolution (Kinect) images

Object segmentation is a fundamental research topic in computer vision. While, only the color information for object segmentation has been the main focus of research, with the availability of low cost color plus range sensors, depth segmentation is now attracting significant attention. This paper presents a novel algorithm for depth segmentation. The proposed technique exploits the divergence of the 2D vector field to segment three-dimensional (3D) object in the depth maps. For a given depth image acquired using a low resolution Kinect sensor, a 2D vector field is computed first at each point of the range image. The depth map is then converted to the div map by computing the 2D vector fields divergence. The latter maps the vector field to a scalar field. The variation of divergence values over the surface contour of the 3D object helps to extract its boundaries. Finally, the depth segmentation is accomplished by applying a threshold to the div map to segment 3D object from the background. In addition to removing the background, the proposed technique also segments the object from the surface on which the object is positioned. The proposed technique was tested on low resolution Washington RGB-D (Kinect) object dataset. Preliminary experimental results suggest that the proposed algorithm achieves better depth segmentation compared to state-of-the art graph-based depth segmentation. The proposed technique also outperforms the latter by achieving 40% higher computational efficiency.

A Guide to Convolutional Neural Networks for Computer Vision

Computer vision has become increasingly important and effective in recent years due to its wide-ranging applications in areas as diverse as smart surveillance and monitoring, health and medicine, sports and recreation, robotics, drones, and self-driving cars. Visual recognition tasks, such as image classification, localization, and detection, are the core building blocks of many of these applications, and recent developments in Convolutional Neural Networks (CNNs) have led to outstanding performance in these state-of-the-art visual recognition tasks and systems. As a result, CNNs now form the crux of deep learning algorithms in computer vision. This self-contained guide will benefit those who seek to both understand the theory behind CNNs and to gain hands-on experience on the application of CNNs in computer vision. It provides a comprehensive introduction to CNNs starting with the essential concepts behind neural networks: training, regularization, and optimization of CNNs. The book also discusses a wide range of loss functions, network layers, and popular CNN architectures, reviews the different techniques for the evaluation of CNNs, and presents some popular CNN tools and libraries that are commonly used in computer vision. Further, this text describes and discusses case studies that are related to the application of CNN in computer vision, including image classification, object detection, semantic segmentation, scene understanding, and image generation. This book is ideal for undergraduate and graduate students, as no prior background knowledge in the field is required to follow the material, as well as new researchers, developers, engineers, and practitioners who are interested in gaining a quick understanding of CNN models.

Efficient Image Set Classification Using Linear Regression Based Image Reconstruction

We propose a novel image set classification technique using linear regression models. Downsampled gallery image sets are interpreted as subspaces of a high dimensional space to avoid the computationally expensive training step. We estimate regression models for each test image using the class specific gallery subspaces. Images of the test set are then reconstructed using the regression models. Based on the minimum reconstruction error between the reconstructed and the original images, a weighted voting strategy is used to classify the test set. We performed extensive evaluation on the benchmark UCSD/Honda, CMU Mobo and YouTube Celebrity datasets for face classification, and ETH-80 dataset for object classification. The results demonstrate that by using only a small amount of training data, our technique achieved competitive classification accuracy and superior computational speed compared with the state-of-the-art methods.