Shankar M. Venkatesan

Quantifiable fitness tracking using wearable devices.

Monitoring health and fitness is emerging as an important benefit that smartphone users could expect from their mobile devices today. Rule of thumb calorie tracking and recommendation based on selective activity monitoring is widely available today, as both on-device and server based solutions. What is surprisingly not available to the users is a simple application geared towards quantitative fitness tracking. Such an application potentially can be a direct indicator of ones cardio-vascular performance and associated long term health risks. Since wearable devices with various inbuilt sensors like accelerometer, gyroscope, SPO2 and heart rate are increasingly becoming available, it is vital that the enormous data coming from these sensors be used to perform analytics to uncover hidden health and fitness associated facts. A continuous estimation of fitness level employing these wearable devices can potentially help users in setting personalized short and long-term exercise goals leading to positive impact on ones overall health. The present work describes a step in this direction. This work involves an unobtrusive method to track an individuals physical activity seamlessly, estimate calorie consumption during a day by mapping the activity to the calories spent and assess fitness level using heart rate data from wearable sensors. We employ a heart rate based parameter called Endurance to quantitatively estimate cardio-respiratory fitness of a person. This opens up avenues for personalization and adaptiveness by dynamically using individuals personal fitness data towards building robust modeling based on analytical principles.

Remote health monitoring system for detecting cardiac disorders.

Remote health monitoring system with Clinical Decision Support System as a key component could potentially quicken the response of medical specialists to critical health emergencies experienced by their patients. A monitoring system, specifically designed for cardiac care with ECG signal analysis as the core diagnostic technique, could play a vital role in early detection of a wide range of cardiac ailments, from a simple arrhythmia to life threatening conditions such as Myocardial Infarction. The system, that we have developed consists of three major components viz., (a) Mobile Gateway, deployed on patients mobile device, that receives 12-Lead ECG signals from any ECG sensor (b) remote server component that hosts algorithms for accurate annotation and analysis of the ECG signal and (c) Point of Care Device of the doctor to receive a diagnostic report from the server based on the analysis of ECG signals. In the present work our focus has been towards developing a system capable of detecting critical cardiac events well in advance using an advanced remote monitoring system. A system of this kind is expected to have applications ranging from tracking wellness/fitness to detection of symptoms leading to fatal cardiac events.

Audio Visual Speech Recognition Using Deep Recurrent Neural Networks

In this work, we propose a training algorithm for an audio-visual automatic speech recognition (AV-ASR) system using deep recurrent neural network (RNN). First, we train a deep RNN acoustic model with a Connectionist Temporal Classification (CTC) objective function. The frame labels obtained from the acoustic model are then used to perform a non-linear dimensionality reduction of the visual features using a deep bottleneck network. Audio and visual features are fused and used to train a fusion RNN. The use of bottleneck features for visual modality helps the model to converge properly during training. Our system is evaluated on GRID corpus. Our results show that presence of visual modality gives significant improvement in character error rate (CER) at various levels of noise even when the model is trained without noisy data. We also provide a comparison of two fusion methods: feature fusion and decision fusion.

Fast Non-blind Image Deblurring with Sparse Priors

Capturing clear images in dim light conditions remains a critical problem in digital photography. Long exposure time inevitably leads to motion blur due to camera shake. On the other hand, short exposure time with high gain yields sharp but noisy images. However, exploiting information from both the blurry and noisy images can produce superior results in image reconstruction. In this paper, we employ the image pairs to carry out a non-blind deconvolution and compare the performances of three different deconvolution methods, namely, Richardson Lucy algorithm, Algebraic deconvolution, and Basis Pursuit deconvolution. We show that the Basis Pursuit approach produces the best results in most cases.

Accommodating Phonetic Word Variations Through Generated Confusion Pairs for Hinglish Handwritten Text Recognition

On-line handwriting recognition has seen major strides in the past years, especially with the advent of deep learning techniques. Recent work has seen the usage of deep networks for sequential classification of unconstrained handwriting recognition task. However, the recognition of “Hinglish” language faces various unseen problems. Hinglish is a portmanteau of Hindi and English, involving frequent code-switching between the two languages. Millions of Indians use Hinglish as a primary mode of communication, especially across social media. However, being a colloquial language, Hinglish does not have a fixed rule set for spelling and grammar. Auto-correction is an unsuitable solution as there is no correct form of the word, and all the multiple phonetic variations are valid. Unlike the advantage that keyboards provide, recognizing handwritten text also has to overcome the issue of mis-recognizing similar looking alphabets. We propose a comprehensive solution to overcome this problem of recognizing words with phonetic spelling variations. To our knowledge, no work has been done till date to recognize Hinglish handwritten text. Our proposed solution shows a character recognition accuracy of 94% and word recognition accuracy of 72%, thus correctly recognizing the multiple phonetic variations of any given word.

Low Dimensional Deep Features for facial landmark alignment

We propose a Low-Dimensional Deep Feature based Face Alignment (LDFFA) method to address the problem of face alignment “in-the-wild”. Recently, Deep Bottleneck Features (DBF) has been proposed as an effective channel to represent input with compact, low-dimensional descriptors. The locations of fiducial landmarks of human faces could be effectively represented using low dimensional features due to the large correlation between them. In this paper, we propose a novel deep CNN with a bottleneck layer which learns to extract a low-dimensional representation (DBF) of the fiducial landmarks from images of human faces. We pre-train the CNN with a large dataset of synthetically annotated data so that the extracted DBFs are robust across variations in pose, occlusions, and illumination. Our experiments show that the proposed approach demonstrates near real-time performance and higher accuracy when compared with state-of-the-art results on numerous benchmarks.

Template-Based Automatic High-Speed Relighting of Faces

This paper presents a framework which estimates the surface normals of human face, surface albedo using an average 3D face template and subsequently replaces the original lighting on the face with a novel lighting in real time. The system uses a facial feature tracking algorithm, which locates and estimates the orientation of face. The 2D facial landmarks thus obtained are used to morph the template model to resemble the input face. The lighting conditions are approximated as a linear combination of Spherical Harmonic bases. A photometric refinement is applied to accurately estimate the surface normal and thus surface albedo. A novel skin-mask construction algorithm is also used to restrict the processing to facial region of the input image. The face is relighted with novel illumination parameters. A novel, cost-effective, seamless blending operation is performed to achieve efficacious and realistic outputs. The process is fully automatic and is executed in real time.