Otkrist Gupta

Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging

The recovery of objects obscured by scattering is an important goal in imaging and has been approached by exploiting, for example, coherence properties, ballistic photons or penetrating wavelengths. Common methods use scattered light transmitted through an occluding material, although these fail if the occluder is opaque. Light is scattered not only by transmission through objects, but also by multiple reflection from diffuse surfaces in a scene. This reflected light contains information about the scene that becomes mixed by the diffuse reflections before reaching the image sensor. This mixing is difficult to decode using traditional cameras. Here we report the combination of a time-of-flight technique and computational reconstruction algorithms to untangle image information mixed by diffuse reflection. We demonstrate a three-dimensional range camera able to look around a corner using diffusely reflected light that achieves sub-millimetre depth precision and centimetre lateral precision over 40 cm×40 cm×40 cm of hidden space.

Object classification through scattering media with deep learning on time resolved measurement

We demonstrate an imaging technique that allows identification and classification of objects hidden behind scattering media and is invariant to changes in calibration parameters within a training range. Traditional techniques to image through scattering solve an inverse problem and are limited by the need to tune a forward model with multiple calibration parameters (like camera field of view, illumination position etc.). Instead of tuning a forward model and directly inverting the optical scattering, we use a data driven approach and leverage convolutional neural networks (CNN) to learn a model that is invariant to calibration parameters variations within the training range and nearly invariant beyond that. This effectively allows robust imaging through scattering conditions that is not sensitive to calibration. The CNN is trained with a large synthetic dataset generated with a Monte Carlo (MC) model that contains random realizations of major calibration parameters. The method is evaluated with a time-resolved camera and multiple experimental results are provided including pose estimation of a mannequin hidden behind a paper sheet with 23 correct classifications out of 30 tests in three poses (76.6% accuracy on real-world measurements). This approach paves the way towards real-time practical non line of sight (NLOS) imaging applications.

Real-Time Physiological Measurement and Visualization Using a Synchronized Multi-camera System

Remote physiological measurement has widespread implications in healthcare and affective computing. This paper presents an efficient system for remotely measuring heart rate and heart rate variability using multiple low-cost digital cameras in real-time. We combine an RGB camera, monochrome camera with color filter and a thermal camera to recover the blood volume pulse (BVP). We show that using multiple cameras in synchrony yields the most accurate recovery of the BVP signal. The RGB combination is not optimal. We show that the thermal camera improves performance of measurement under dynamic ambient lighting but the thermal camera alone is not enough and accuracy can be improved by adding more spectral channels. We present a real-time prototype that allows accurate physiological measurement combined with a novel user interface to visualize changes in heart rate and heart rate variability. Finally, we propose how this system might be used for applications such as patient monitoring.

Multi-velocity neural networks for facial expression recognition in videos

We present a new action recognition deep neural network which adaptively learns the best action velocities in addition to the classification. While deep neural networks have reached maturity for image understanding tasks, we are still exploring network topologies and features to handle the richer environment of video clips. Here, we tackle the problem of multiple velocities in action recognition, and provide state-of-the-art results for facial expression recognition, on known and new collected datasets. We further provide the training steps for our semi-supervised network, suited to learn from huge unlabeled datasets with only a fraction of labeled examples.

Machine learning approaches for large scale classification of produce

The analysis and identification of different attributes of produce such as taxonomy, vendor, and organic nature is vital to verifying product authenticity in a distribution network. Though a variety of analysis techniques have been studied in the past, we present a novel data-centric approach to classifying produce attributes. We employed visible and near infrared (NIR) spectroscopy on over 75,000 samples across several fruit and vegetable varieties. This yielded 0.90–0.98 and 0.98–0.99 classification accuracies for taxonomy and farmer classes, respectively. The most significant factors in the visible spectrum were variations in the produce color due to chlorophyll and anthocyanins. In the infrared spectrum, we observed that the varying water and sugar content levels were critical to obtaining high classification accuracies. High quality spectral data along with an optimal tuning of hyperparameters in the support vector machine (SVM) was also key to achieving high classification accuracies. In addition to demonstrating exceptional accuracies on test data, we explored insights behind the classifications, and identified the highest performing approaches using cross validation. We presented data collection guidelines, experimental design parameters, and machine learning optimization parameters for the replication of studies involving large sample sizes.

Illumination invariants in deep video expression recognition

Abstract In this paper we present architectures based on deep neural nets for expression recognition in videos, which are invariant to local scaling. We amalgamate autoencoder and predictor architectures using an adaptive weighting scheme coping with a reduced size labeled dataset, while enriching our models from enormous unlabeled sets. We further improve robustness to lighting conditions by introducing a new adaptive filter based on temporal local scale normalization. We provide superior results over known methods, including recent reported approaches based on neural nets.

Pairwise Confusion for Fine-Grained Visual Classification

Fine-Grained Visual Classification (FGVC) datasets contain small sample sizes, along with significant intra-class variation and inter-class similarity. While prior work has addressed intra-class variation using localization and segmentation techniques, inter-class similarity may also affect feature learning and reduce classification performance. In this work, we address this problem using a novel optimization procedure for the end-to-end neural network training on FGVC tasks. Our procedure, called Pairwise Confusion (PC) reduces overfitting by intentionally introducing confusion in the activations. With PC regularization, we obtain state-of-the-art performance on six of the most widely-used FGVC datasets and demonstrate improved localization ability. PC is easy to implement, does not need excessive hyperparameter tuning during training, and does not add significant overhead during test time.

Distributed learning of deep neural network over multiple agents

In domains such as health care and finance, shortage of labeled data and computational resources is a critical issue while developing machine learning algorithms. To address the issue of labeled data scarcity in training and deployment of neural network-based systems, we propose a new technique to train deep neural networks over several data sources. Our method allows for deep neural networks to be trained using data from multiple entities in a distributed fashion. We evaluate our algorithm on existing datasets and show that it obtains performance which is similar to a regular neural network trained on a single machine. We further extend it to incorporate semi-supervised learning when training with few labeled samples, and analyze any security concerns that may arise. Our algorithm paves the way for distributed training of deep neural networks in data sensitive applications when raw data may not be shared directly.

Technology-enabled examinations of cardiac rhythm, optic nerve, oral health, tympanic membrane, gait and coordination evaluated jointly with routine health screenings: an observational study at the 2015 Kumbh Mela in India

Objectives Technology-enabled non-invasive diagnostic screening (TES) using smartphones and other point-of-care medical devices was evaluated in conjunction with conventional routine health screenings for the primary care screening of patients. Design Dental conditions, cardiac ECG arrhythmias, tympanic membrane disorders, blood oxygenation levels, optic nerve disorders and neurological fitness were evaluated using FDA-approved advanced smartphone powered technologies. Routine health screenings were also conducted. A novel remote web platform was developed to allow expert physicians to examine TES data and compare efficacy with routine health screenings. Setting The study was conducted at a primary care centre during the 2015 Kumbh Mela in Maharashtra, India. Participants 494 consenting 18–90 years old adults attending the 2015 Kumbh Mela were tested. Results TES and routine health screenings identified unique clinical conditions in distinct patients. Intraoral fluorescent imaging classified 63.3% of the population with dental caries and periodontal diseases. An association between poor oral health and cardiovascular illnesses was also identified. Tympanic membrane imaging detected eardrum abnormalities in 13.0% of the population, several with a medical history of hearing difficulties. Gait and coordination issues were discovered in eight subjects and one subject had arrhythmia. Cross-correlations were observed between low oxygen saturation and low body mass index (BMI) with smokers (p=0.0087 and p=0.0122, respectively), and high BMI was associated with elevated blood pressure in middle-aged subjects. Conclusions TES synergistically identified clinically significant abnormalities in several subjects who otherwise presented as normal in routine health screenings. Physicians validated TES findings and used routine health screening data and medical history responses for comprehensive diagnoses for at-risk patients. TES identified high prevalence of oral diseases, hypertension, obesity and ophthalmic conditions among the middle-aged and elderly Indian population, calling for public health interventions.

RFlow: User Interaction Beyond Walls

Current user-interaction with optical gesture tracking technologies suffer from occlusions, limiting the functionality to direct line-of-sight. We introduce RFlow, a compact, medium-range interface based on Radio Frequency (RF) that enables camera-free tracking of the position of a moving hand through drywall and other occluders. Our system uses Time of Flight (TOF) RF sensors and speed-based segmentation to localize the hand of a single user with 5cm accuracy (as measured to the closest ground-truth point), enabling an interface which is not restricted to a training set.