Arun C. S. Kumar

A multiscale approach for recognizing complex annotations in engineering documents

A novel method for character recognition targeted at complex annotations found in engineering documents is presented. A feasibility study is described in which characters extracted from engineering drawings were recognized without error from a class of 36 distinct alphanumeric patterns by a neural network classifier trained with multiscale representations. An incremental strategy is presented for resolution which relies upon the continuity between hierarchical levels of a novel multiscale decomposition. The authors observed a 16-fold reduction in the amount of information needed to represent each character for recognition. These results suggest high reliability at a reduced cost of representation.<<ETX>>

Joint geometric graph embedding for partial shape matching in images

A novel multi-criteria optimization framework for matching of partially visible shapes in multiple images using joint geometric graph embedding is proposed. The proposed framework achieves matching of partial shapes in images that exhibit extreme variations in scale, orientation, viewpoint and illumination and also instances of occlusion; conditions which render impractical the use of global contour-based descriptors or local pixel-level features for shape matching. The proposed technique is based on optimization of the embedding distances of geometric features obtained from the eigenspectrum of the joint image graph, coupled with regularization over values of the mean pixel intensity or histogram of oriented gradients. It is shown to obtain successfully the correspondences denoting partial shape similarities as well as correspondences between feature points in the images. A new benchmark dataset is proposed which contains disparate image pairs with extremely challenging variations in viewing conditions when compared to an existing dataset [18]. The proposed technique is shown to significantly outperform several state-of-the-art partial shape matching techniques on both datasets.

A Deep Learning Paradigm for Detection of Harmful Algal Blooms

Effective and cost-efficient monitoring is indispensable for ensuring environmental sustainability. Cyanobacterial Harmful Algal Blooms (CyanoHABs) are a major water quality and public health issue in inland water bodies. The recent popularity of online social media (OSM) platforms coupled with advances in cloud computing and data analytics has given rise to citizen science-based approaches to environmental monitoring. These approaches involve the lay community in the acquisition, collection and transmission of relevant data in the form of tweets, images, voice recordings and videos typically acquired using low-cost mobile devices such as smartphones or tablet computers. While cost effective, citizen science-based approaches are highly susceptible to noise, inaccuracies and missing data. In this paper we address the problem of automated detection of harmful algal blooms (HABs) via analysis of image data of inland water bodies. These image data are acquired using a variety of smartphones and communicated via popular OSM platforms such as Facebook, Twitter and Instagram. To account for the wide variations in imaging parameters and ambient environmental parameters we propose a deep learning approach to image feature extraction and classification for the purpose of HAB detection. The current system is a first step in the design of an automated early detection, warning and rapid response system that can be adopted to mitigate the detrimental effects of CyanoHAB contamination of inland water bodies.

Action Recognition in Still Images Using Word Embeddings from Natural Language Descriptions

Detecting actions or verbs in still images is a challenging problem for a variety of reasons such as the absence of temporal information and polysemy of verbs which lead to difficulty in generating large verb datasets. In this paper, we propose to first detect the prominent objects in the image and then infer the relevant actions or verbs using Natural Language Processing (NLP)-based techniques. The proposed scheme obviates the need for training and using visual action detectors on images, an approach which tends to be error-prone and computationally intensive. This paper provides a valuable insight in that the detection of a few significant (i.e., top) objects in an image allows one to extract or infer the relevant actions or verbs in the image. To this end, we propose NLP-based approaches relying on the word2vec and the Object-Verb-Object triplet models for predicting the actions from top-object detections and also analyze their nuances. Our experimental results show that verbs can be reliably and efficiently detected by exploiting the top object detections in an image.

Guessing objects in context

Large scale object classification has seen commendable progress owing, in large part, to recent advances in deep learning. However, generating annotated training datasets is still a significant challenge, especially when training classifiers for large number of object categories. In these situations, generating training datasets is expensive coupled with the fact that training data may not be available for all categories and situations. Such situations are generally resolved using zero-shot learning. However, training zero-shot classifiers entails serious programming effort and is not scalable to very large number of object categories. We propose a novel simple framework that can guess objects in an image. The proposed framework has the advantages of scalability and ease of use with minimal loss in accuracy. The proposed framework answers the following question: How does one guess objects in an image from very few object detections?

Class-Specific Object Pose Estimation and Reconstruction Using 3D Part Geometry

We propose a novel approach for detecting and reconstructing class-specific objects from 2D images. Reconstruction and detection, despite major advances, are still wanting in performance. Hence, approaches that try to solve them jointly, so that one can be used to resolve the ambiguities of the other, especially while employing data-driven class-specific learning, are increasingly popular. In this paper, we learn a deformable, fine-grained, part-based model from real world, class-specific, image sequences, so that given a new image, we can simultaneously estimate the 3D shape, viewpoint and the subsequent 2D detection results. This is a step beyond existing approaches, which are usually limited to 3D CAD shapes, regression based pose estimation, template based deformation modelling etc. We employ Structure from Motion (SfM) and part based models in our learning process, and estimate a 3D deformable object instance and a projection matrix that explains the image information. We demonstrate our approach with high quality qualitative and quantitative results on our real world RealCar dataset, as well as the EPFL car dataset.