Viswanath Gopalakrishnan

Neutral Face Classification Using Personalized Appearance Models for Fast and Robust Emotion Detection

Facial expression recognition is one of the open problems in computer vision. Robust neutral face recognition in real time is a major challenge for various supervised learning-based facial expression recognition methods. This is due to the fact that supervised methods cannot accommodate all appearance variability across the faces with respect to race, pose, lighting, facial biases, and so on, in the limited amount of training data. Moreover, processing each and every frame to classify emotions is not required, as user stays neutral for majority of the time in usual applications like video chat or photo album/web browsing. Detecting neutral state at an early stage, thereby bypassing those frames from emotion classification would save the computational power. In this paper, we propose a light-weight neutral versus emotion classification engine, which acts as a pre-processer to the traditional supervised emotion classification approaches. It dynamically learns neutral appearance at key emotion (KE) points using a statistical texture model, constructed by a set of reference neutral frames for each user. The proposed method is made robust to various types of user head motions by accounting for affine distortions based on a statistical texture model. Robustness to dynamic shift of KE points is achieved by evaluating the similarities on a subset of neighborhood patches around each KE point using the prior information regarding the directionality of specific facial action units acting on the respective KE point. The proposed method, as a result, improves emotion recognition (ER) accuracy and simultaneously reduces computational complexity of the ER system, as validated on multiple databases.

Fast human segmentation using color and depth

Accurate segmentation of humans from live videos is an important problem to be solved in developing immersive video experience. We propose to extract the human segmentation information from color and depth cues in a video using multiple modeling techniques. The prior information from human skeleton data is also fused along with the depth and color models to obtain the final segmentation inside a graph-cut framework. The proposed method runs real time on live videos using single CPU and is shown to be quantitatively outperforming the methods that directly fuse color and depth data.

Similarity and rigidity preserving image retargeting

Conventional Image Retargeting methods aim to preserve the salient regions in an image using As Similar as Possible (ASAP) energy formulation or As Rigid as Possible (ARAP) energy formulation. ASAP energy formulation preserves the shape of the salient object while the scale of salient object can get distorted in the retargeted image. On the contrary, ARAP energy formulation preserves the scale of the salient object while the shape can get compromised. We propose a novel technique in which the necessity to preserve similarity and rigidity of the salient object is taken into consideration in a single energy formulation. The concept of object centric saliency is also introduced to improve the quality of image retargeting. The subjective and objective results on retargetMe database demonstrate the advantage of the proposed method than using ASAP or ARAP energy alone.

Learning rotation invariance in deep hierarchies using circular symmetric filters

Deep hierarchical models for feature learning have emerged as an effective technique for object representation and classification in recent years. Though the features learnt using deep models have shown lot of promise towards achieving invariance to data transformations, this primarily comes at the expense of using much larger training data and model size. In the proposed work we devise a novel technique to incorporate rotation invariance, while training the deep model parameters. The convolution weight parameters in the network architecture are constrained to exhibit circular symmetry resulting in “rotation equivariance” of output feature maps. Rotation invariance is further achieved by max-pooling of the feature maps later in the hierarchy. We also show that by incorporating circular symmetry constraint into the training loss function, rotation invariance can be achieved with-in deep neural network framework with much lesser training data and model parameters. Our experiment results evaluated on rotated MNIST dataset further objectively validate the contribution.

Real-time video summarization on mobile

This paper proposes a novel approach for real-time video summarization on mobile using Dictionary Learning, Global Camera Motion analysis and Colorfulness. A dictionary is represented as a distinct set of events that are described as spatio-temporal features. Uniqueness measure is predicted based on the correlation scores of the dictionary elements whereas the quality measure is estimated using Global Camera Motion analysis and Colorfulness. Our proposed technique combines the uniqueness measure and the quality measure to predict the interestingness. Experiments indicate that our method outperforms state-of-the-arts in terms of computation speed while retaining the similar subjective quality.