Konda Reddy Mopuri

Object level deep feature pooling for compact image representation

Convolutional Neural Network (CNN) features have been successfully employed in recent works as an image descriptor for various vision tasks. But the inability of the deep CNN features to exhibit invariance to geometric transformations and object compositions poses a great challenge for image search. In this work, we demonstrate the effectiveness of the objectness prior over the deep CNN features of image regions for obtaining an invariant image representation. The proposed approach represents the image as a vector of pooled CNN features describing the underlying objects. This representation provides robustness to spatial layout of the objects in the scene and achieves invariance to general geometric transformations, such as translation, rotation and scaling. The proposed approach also leads to a compact representation of the scene, making each image occupy a smaller memory footprint. Experiments show that the proposed representation achieves state of the art retrieval results on a set of challenging benchmark image datasets, while maintaining a compact representation.

Towards semantic visual representation: augmenting image representation with natural language descriptors

Learning image representations has been an interesting and challenging problem. When users upload images to photo sharing websites, they often provide multiple textual tags for ease of reference. These tags can reveal significant information about the content of the image such as the objects present in the image or the action that is taking place. Approaches have been proposed to extract additional information from these tags in order to augment the visual cues and build a multi-modal image representation. However, the existing approaches do not pay much attention to the semantic meaning of the tags while they encode. In this work, we attempt to enrich the image representation with the tag encodings that leverage their semantics. Our approach utilizes neural network based natural language descriptors to represent the tag information. By complementing the visual features learned by convnets, our approach results in an efficient multi-modal image representation. Experimental evaluation suggests that our approach results in a better multi-modal image representation by exploiting the two data modalities for classification on benchmark datasets.

An Introduction to Deep Convolutional Neural Nets for Computer Vision

Abstract Traditional architectures for solving computer vision problems and the degree of success they enjoyed have been heavily reliant on hand-crafted features. However, of late, deep learning techniques have offered a compelling alternative – that of automatically learning problem-specific features. With this new paradigm, every problem in computer vision is now being re-examined from a deep learning perspective. Therefore, it has become important to understand what kind of deep networks are suitable for a given problem. We specifically consider one form of deep networks widely used in computer vision – convolutional neural networks (CNNs). We start with “AlexNet” as our base CNN and then examine the broad variations proposed over time for many applications. We hope that our recipe-style presentation will serve as a guide, particularly for novice practitioners intending to use deep-learning techniques for computer vision.