Eric C. Larson

Most apparent distortion: full-reference image quality assessment and the role of strategy

The mainstream approach to image quality assessment has centered around accurately modeling the single most relevant strategy employed by the human visual system (HVS) when judging image quality (e.g., detecting visible differences, and extracting image structure/information). In this work, we suggest that a single strategy may not be sufficient; rather, we advocate that the HVS uses multiple strategies to determine image quality. For images containing near-threshold distortions, the image is most apparent, and thus the HVS attempts to look past the image and look for the distortions (a detection-based strategy). For images containing clearly visible distortions, the distortions are most apparent, and thus the HVS attempts to look past the distortion and look for the images subject matter (an appearance-based strategy). Here, we present a quality assessment method [most apparent distortion (MAD)], which attempts to explicitly model these two separate strategies. Local luminance and contrast masking are used to estimate detection-based perceived distortion in high-quality images, whereas changes in the local statistics of spatial-frequency components are used to estimate appearance-based perceived distortion in low-quality images. We show that a combination of these two measures can perform well in predicting subjective ratings of image quality.

Accurate and privacy preserving cough sensing using a low-cost microphone

Audio-based cough detection has become more pervasive in recent years because of its utility in evaluating treatments and the potential to impact the quality of life for individuals with chronic cough. We critically examine the current state of the art in cough detection, concluding that existing approaches expose private audio recordings of users and bystanders. We present a novel algorithm for detecting coughs from the audio stream of a mobile phone. Our system allows cough sounds to be reconstructed from the feature set, but prevents speech from being reconstructed intelligibly. We evaluate our algorithm on data collected in the wild and report an average true positive rate of 92% and false positive rate of 0.5%. We also present the results of two psychoacoustic experiments which characterize the tradeoff between the fidelity of reconstructed cough sounds and the intelligibility of reconstructed speech.

SpiroSmart: using a microphone to measure lung function on a mobile phone

Home spirometry is gaining acceptance in the medical community because of its ability to detect pulmonary exacerbations and improve outcomes of chronic lung ailments. However, cost and usability are significant barriers to its widespread adoption. To this end, we present SpiroSmart, a low-cost mobile phone application that performs spirometry sensing using the built-in microphone. We evaluate SpiroSmart on 52 subjects, showing that the mean error when compared to a clinical spirometer is 5.1% for common measures of lung function. Finally, we show that pulmonologists can use SpiroSmart to diagnose varying degrees of obstructive lung ailments.

DopLink: using the doppler effect for multi-device interaction

Mobile and embedded electronics are pervasive in todays environment. As such, it is necessary to have a natural and intuitive way for users to indicate the intent to connect to these devices from a distance. We present DopLink, an ultrasonic-based device selection approach. It utilizes the already embedded audio hardware in smart devices to determine if a particular device is being pointed at by another device (i.e., the user waves their mobile phone at a target in a pointing motion). We evaluate the accuracy of DopLink in a controlled user study, showing that, within 3 meters, it has an average accuracy of 95% for device selection and 97% for finding relative device position. Finally, we show three applications of DopLink: rapid device pairing, home automation, and multi-display synchronization.

Dante vision: In-air and touch gesture sensing for natural surface interaction with combined depth and thermal cameras

Researchers have paid considerable attention to natural user interfaces, especially sensing gestures and touches upon an un-instrumented surface from an overhead camera. We present a system that combines depth sensing from a Microsoft Kinect and temperature sensing from a thermal imaging camera to infer a variety of gestures and touches for controlling a natural user interface. The system, coined Dante, is capable of (1) inferring multiple touch points from multiple users (92.6% accuracy), (2) detecting and classifying each user using their depth and thermal footprint (87.7% accuracy), and (3) detecting touches on objects placed upon the table top (91.7% accuracy). The system can also classify the pressure of chording motions. The system is real time, with an average processing delay of 40 ms.

SpiroCall: Measuring Lung Function over a Phone Call

Cost and accessibility have impeded the adoption of spirometers (devices that measure lung function) outside clinical settings, especially in low-resource environments. Prior work, called SpiroSmart, used a smartphones built-in microphone as a spirometer. However, individuals in low- or middle-income countries do not typically have access to the latest smartphones. In this paper, we investigate how spirometry can be performed from any phone-using the standard telephony voice channel to transmit the sound of the spirometry effort. We also investigate how using a 3D printed vortex whistle can affect the accuracy of common spirometry measures and mitigate usability challenges. Our system, coined SpiroCall, was evaluated with 50 participants against two gold standard medical spirometers. We conclude that SpiroCall has an acceptable mean error with or without a whistle for performing spirometry, and advantages of each are discussed.

DOSE: Detecting user-driven operating states of electronic devices from a single sensing point

Electricity and appliance usage information can often reveal the nature of human activities in a home. For instance, sensing the use of vacuum cleaner, a microwave oven, and kitchen appliances can give insights into a persons current activities. Instead of putting a sensor on each appliance, our technique is based on the idea that appliance usage can be sensed by their manifestations in an environments existing electrical infrastructure. Prior approaches using this technique could only detect an appliances on-off states; that is, they only sense “what” is being used, but not “how” it is used. In this paper, we introduce DOSE, a significant advancement for inferring operating states of electronic devices from a single sensing point in a home. When an electronic device is in operation, it generates time-varying Electromagnetic Interference (EMI) based upon its operating states (e.g., vacuuming on a rug vs. hardwood floor). This EMI noise is coupled to the power line and can be picked up from a single sensing hardware attached to the wall outlet in a house. Unlike prior data-driven approaches, we employ domain knowledge of the devices circuitry for semi-supervised model training to avoid tedious labeling process. We evaluated DOSE in a residential house for 2 months and found that operating states for 16 appliances could be estimated with an average accuracy of 93.8%. These fine-grained electrical characteristics affords rich feature sets of electrical events and have the potential to support various applications such as in-home activity inference, energy disaggregation and device failure detection.

PupilWare: towards pervasive cognitive load measurement using commodity devices

Cognitive load refers to the amount of effort required by an individual to process information. Dating back more than fifty years, the cognitive psychology community has conducted experiments showing that the cognitive load experienced by an individual can be measured using sub-millimeter fluctuations in their pupil size, assessed using medical grade infrared devices known as pupillometers, and more recently, infrared eye-trackers. However the cost and availability of these eye-trackers limits most pupil response measurement to laboratory settings. We argue that ubiquitously measuring pupillary response could transform the next generation of context aware computing applications---enabling computational devices to understand a users current ability to process information, especially for users with cognitive disabilities. To this end, we present PupilWare, a system that analyzes pupil size changes through commodity cameras like those in a laptop. We evaluate PupilWares ability to measure changes in pupil dilation using classic cognitive psychology experiments and validate its performance compared to infrared gaze trackers and medical grade pupillometers. We conclude that, in controlled conditions, PupilWare is as accurate as infrared eye-tracking for assessing task evoked cognitive load, though has problems with dark eyed individuals and eyelid occlusion.

Performance-analysis-based acceleration of image quality assessment

Two stages are commonly employed in modern algorithms of image/video quality assessment (QA): (1) a local frequency-based decomposition, and (2) block-based statistical comparisons between the frequency coefficients of the reference and distorted images. This paper presents a performance analysis of and techniques for accelerating these stages. We specifically analyze and accelerate one representative QA algorithm recently developed by the authors (Larson and Chandler, 2010). We identify the bottlenecks from the abovementioned stages, and we present methods of acceleration using integral images, inline expansion, a GPGPU implementation, and other code modifications. We show how a combination of these approaches can yield a speedup of 47×.

Use of a Smartphone App to Assess Neonatal Jaundice

The accuracy of a new technology based on the analysis of digital images obtained by using a smartphone app in estimating bilirubin levels in newborns is assessed. BACKGROUND: The assessment of jaundice in outpatient neonates is problematic. Visual assessment is inaccurate, and more exact methodologies are cumbersome and/or expensive. Our goal in this study was to assess the accuracy of a technology based on the analysis of digital images of newborns obtained using a smartphone application called BiliCam. METHODS: Paired BiliCam images and total serum bilirubin (TSB) levels were obtained in a diverse sample of newborns (<7 days old) at 7 sites across the United States. By using specialized software, data on color values in the images (“features”) were extracted. Machine learning and regression analysis techniques were used to identify features for inclusion in models to predict an estimated bilirubin level for each newborn. The correlation between estimated bilirubin levels and TSB levels was calculated. In addition, the sensitivity and specificity of the estimated bilirubin levels in identifying newborns with high TSB levels were calculated by using 2 recommended decision rules for jaundice screening. RESULTS: Estimated bilirubin levels were calculated and compared with TSB levels in a diverse sample of 530 newborns (20.8% African American, 26.3% Hispanic, and 21.2% Asian American). The overall correlation was 0.91, and correlations among white, African American, Hispanic, and Asian American newborns were 0.92, 0.90, 0.91, and 0.88, respectively. The sensitivities of BiliCam in identifying newborns with high TSB levels were 84.6% and 100%, respectively, by using 2 decision rules; specificities were 75.1% and 76.4%, respectively. CONCLUSIONS: BiliCam provided accurate estimates of TSB values, demonstrating that an inexpensive technology that uses commodity smartphones could be used to effectively screen newborns for jaundice.