Shankar Setty

Indian Movie Face Database: A benchmark for face recognition under wide variations

Recognizing human faces in the wild is emerging as a critically important, and technically challenging computer vision problem. With a few notable exceptions, most previous works in the last several decades have focused on recognizing faces captured in a laboratory setting. However, with the introduction of databases such as LFW and Pubfigs, face recognition community is gradually shifting its focus on much more challenging unconstrained settings. Since its introduction, LFW verification benchmark is getting a lot of attention with various researchers contributing towards state-of-the-results. To further boost the unconstrained face recognition research, we introduce a more challenging Indian Movie Face Database (IMFDB) that has much more variability compared to LFW and Pubfigs. The database consists of 34512 faces of 100 known actors collected from approximately 103 Indian movies. Unlike LFW and Pubfigs which used face detectors to automatically detect the faces from the web collection, faces in IMFDB are detected manually from all the movies. Manual selection of faces from movies resulted in high degree of variability (in scale, pose, expression, illumination, age, occlusion, makeup) which one could ever see in natural world. IMFDB is the first face database that provides a detailed annotation in terms of age, pose, gender, expression, amount of occlusion, for each face which may help other face related applications.

Framework for 3D object hole filling

In this paper we address the problem of hole filling in a point cloud of 3D object. Even with most popular 3D scanning devices like Microsoft Kinect and Time of Flight (ToF) cameras, occlusions during the scanning process result in occurrence of missing regions or holes in 3D data. We propose a framework for hole filling in a point cloud of 3D object using Riemannian metric tensor and Christoffel symbols as a set of geometric features, which capture the inherent geometry of the 3D object. The framework involves detection and extraction of the boundary points surrounding the hole, decomposition of boundary points into basic shapes and selective surface interpolation to fill the hole. We demonstrate the performance of the proposed method on point clouds with different complexities and sizes for both synthetically generated holes and real missing regions during the capturing process on 3D models of heritage sites.

3D Object Super Resolution using Metric Tensor and Christoffel Symbols

In this paper we address the problem of 3D super resolution. 3D super resolution is a process of generating high resolution point cloud, given a low resolution point cloud. We model 3D object as a set of Riemannian manifolds in continuous and discretized space. We propose to use Riemannian metric tensor and Christoffel symbols as a set of features to capture the inherent geometry of the 3D object. We propose a learning framework to decompose 3D object using metric tensor and Christoffel symbols into a set of basis functions to selectively super resolve the 3D object. We demonstrate the proposed algorithm on 3D objects and achieve better results than reported in literature.

Classification of facebook news feeds and sentiment analysis

As recently seen in Googles Gmail, the messages in inbox are classified into primary, social and promotions, which makes it easy for the users to differentiate the messages which they are looking for from the bulk of messages. Similarly, a users wall in facebook is usually flooded with huge amount of data which makes it annoying for the users to view the important news feeds among the rest. Thus we aim to focuses on classification of facebook news feeds. In this paper, we attempt to classify the users news feeds into various categories using classifiers to provide a better representation of data on users wall. News feeds collected from facebook are dynamically classified into various classes such as friends posts and liked pages posts. Friends posts are further categorized into life events posts and entertainment posts. Posts or updates from pages which are liked by the users are grouped as liked pages posts. Posts from friends are tagged as friends posts and those regarding the events occurring in their lives are said to be life event posts and the rest are tagged as entertainment posts. This helps users to find “important news feeds” from “live news feeds”. Sentiments are important as they depict the opinions and expressions of the user. Hence, detecting the sentiments of users from the life event posts also becomes an essential task. We also propose a system for automatic detection of sentiments from the life event posts and categorize based on sentiments into happy, neutral and bad feelings posts. This paper looks towards applying the classification methods from the literature to our dataset with the objective of evaluating methods of automatic news feeds classification and sentiment analysis which in future can provide facebook page a well organized and more appealing look.

Example-based 3D inpainting of point clouds using metric tensor and Christoffel symbols

In this paper, we address the problem of 3D inpainting using example-based methods for point cloud data. 3D inpainting is a process of filling holes or missing regions in the reconstructed 3D models. Typically inpainting methods addressed in the literature fill missing regions due to occlusions or inaccurate scanning of 3D models. However, we focus on scenarios involving naturally existing damaged models which are partly broken or incomplete in artifacts at cultural heritage sites. We propose two example-based inpainting techniques, namely region of interest (ROI)-based and patch-based methods, to inpaint the missing regions of the damaged model. For both the methods, we represent the 3D model as a set of Riemannian manifolds in Euclidean space, to capture the inherent geometry using metric tensor and Christoffel symbols as geometric features and decompose into basic shape (such as spherical, conical and cylindrical) regions using decomposition algorithm derived from supervised learning. In ROI-based method, instead of using single similar example for inpainting, we select the most relevant regions that best-fit the missing region from the set of basic shape regions derived from n similar examples. And in patch-based method, we not only select the most relevant regions but cluster the regions into a set of patches. The best corresponding patches that match the missing region to be inpainted are considered to be the most relevant best-fit patches that cover the complete missing region. We demonstrate the performance of proposed inpainting methods on cultural heritage artifacts with varying complexities and sizes for both synthetically generated holes and real missing regions.

Region of Interest-Based 3D Inpainting of Cultural Heritage Artifacts

In this article, we address the problem of 3D inpainting using an exemplar-based method for point clouds. 3D inpainting is a process of filling holes or missing regions in the reconstructed 3D models. Typically, inpainting methods addressed in the literature fill missing regions due to occlusions or inaccurate scanning of 3D models. However, we focus on scenarios involving naturally existing damaged models, which are partly broken or incomplete in the artifacts at cultural heritage sites. We propose an exemplar-based inpainting technique using the region of interest (ROI)-based method to inpaint the missing regions of the damaged model. The ROI of a 3D model is represented as a set of Riemannian manifolds, and metric tensor and Christoffel symbols are used as geometric features to capture the inherent geometry. We then decompose the ROI into basic shape regions, namely, spherical, conical, and cylindrical components, and identify the best-fit match for inpainting. Instead of using a single similar exemplar for inpainting, we select the most relevant best-fit region to fill the missing region from the basic shape regions library obtained from n similar exemplars. We demonstrate the performance of the proposed inpainting method on artifacts at UNESCO World Heritage site Hampi temples, India with varying complexities and sizes for both synthetically generated holes and real missing regions in 3D objects.

Restaurant setup business analysis using yelp dataset

In this paper, we address the issues associated with setting-up of a new restaurant business. To strategize a new restaurant business, we propose a restaurant business framework which comprises of 3 most important tasks, namely, high frequency attributes, most crowded day and location of the restaurant. First, we identify the features/attributes of the restaurants in which the customers are most interested in and provide those facilities and services to increase the profit. Next, we identify the day of the week when the restaurants are heavily crowded so that the best recipes and offers are made available on those days. Finally, since location has a profound effect on the success of a restaurant business, we consider location to be the most important to know the nearby restaurants and their facilities before coming up with a new restaurant business. The performance analysis of the proposed framework was carried out on the standard Yelp dataset. Thus, we found credit card to be the most preferred attribute, the most crowded day to be Monday and Divey to be the most desired ambience among the customers. We also demonstrate how the new restaurant can be setup by identifying the nearest restaurants and the services.

Realistic Walkthrough of Cultural Heritage Sites

In this chapter, we present the framework for realistic walkthrough of cultural heritage sites. The framework includes 3D data acquisition, data processing, and interactive rendering of complex 3D models such as sculptures, monuments, and artifacts found at cultural heritage sites. We acquire both coarse level and detail level 3D data using modeling tools and scanning devices. The acquired point cloud data at cultural heritage sites exhibit nonuniform distribution of geometry and hence we propose to use intrinsic geometric properties like metric tensor and Christoffel symbols, for capturing the geometry of the acquired 3D data to facilitate data processing. We propose several geometry-based data processing techniques such as super resolution, hole filling, and object categorization, for refining the acquired 3D data. We also propose coarse to detail 3D reconstruction technique, for the reconstruction of 3D models. Finally, the coarse to detail 3D reconstructed models is rendered using a rendering engine in an attempt to restore the original appearance of cultural heritage sites. We demonstrate the proposed framework using a walkthrough generated for the Vittala Temple at Hampi.

Region of interest (ROI) based 3D inpainting

We address the problem of 3D inpainting using ROI-based method for point cloud data. We focus on inpainting of complex, irregular and large missing regions covering prominent geometric features by considering n self-similar examples. The effectiveness of the proposed framework is demonstrated on 3D artifacts obtained from archaeological sites.

3D object decomposition and super resolution

In this paper we propose to address the problem of 3D object decomposition and super resolution. We model the 3D object as a set of Riemannian manifolds and propose metric tensor and Christoffel symbols as a novel set of features for 3D object decomposition using polynomial kernel SVM classifier. The super resolution of the 3D point clouds is carried out using the decomposed object by using selective interpolation techniques. The effectiveness of the proposed framework is demonstrated on 3D objects obtained from different datasets and achieve comparable results.