Mashfiqui Rabbi

NeuroPhone: brain-mobile phone interface using a wireless EEG headset

Neural signals are everywhere just like mobile phones. We propose to use neural signals to control mobile phones for hands-free, silent and effortless human-mobile interaction. Until recently, devices for detecting neural signals have been costly, bulky and fragile. We present the design, implementation and evaluation of the NeuroPhone system, which allows neural signals to drive mobile phone applications on the iPhone using cheap off-the-shelf wireless electroencephalography (EEG) headsets. We demonstrate a brain-controlled address book dialing app, which works on similar principles to P300-speller brain-computer interfaces: the phone flashes a sequence of photos of contacts from the address book and a P300 brain potential is elicited when the flashed photo matches the person whom the user wishes to dial. EEG signals from the headset are transmitted wirelessly to an iPhone, which natively runs a lightweight classifier to discriminate P300 signals from noise. When a persons contact-photo triggers a P300, his/her phone number is automatically dialed. NeuroPhone breaks new ground as a brain-mobile phone interface for ubiquitous pervasive computing. We discuss the challenges in making our initial prototype more practical, robust, and reliable as part of our on-going research.

Passive and In-Situ assessment of mental and physical well-being using mobile sensors

The idea of continuously monitoring well-being using mobile-sensing systems is gaining popularity. In-situ measurement of human behavior has the potential to overcome the short comings of gold-standard surveys that have been used for decades by the medical community. However, current sensing systems have mainly focused on tracking physical health; some have approximated aspects of mental health based on proximity measurements but have not been compared against medically accepted screening instruments. In this paper, we show the feasibility of a multi-modal mobile sensing system to simultaneously assess mental and physical health. By continuously capturing fine-grained motion and privacy-sensitive audio data, we are able to derive different metrics that reflect the results of commonly used surveys for assessing well-being by the medical community. In addition, we present a case study that highlights how errors in assessment due to the subjective nature of the responses could potentially be avoided by continuous mobile sensing.

Automated Personalized Feedback for Physical Activity and Dietary Behavior Change With Mobile Phones: A Randomized Controlled Trial on Adults

Background A dramatic rise in health-tracking apps for mobile phones has occurred recently. Rich user interfaces make manual logging of users’ behaviors easier and more pleasant, and sensors make tracking effortless. To date, however, feedback technologies have been limited to providing overall statistics, attractive visualization of tracked data, or simple tailoring based on age, gender, and overall calorie or activity information. There are a lack of systems that can perform automated translation of behavioral data into specific actionable suggestions that promote healthier lifestyle without any human involvement. Objective MyBehavior, a mobile phone app, was designed to process tracked physical activity and eating behavior data in order to provide personalized, actionable, low-effort suggestions that are contextualized to the user’s environment and previous behavior. This study investigated the technical feasibility of implementing an automated feedback system, the impact of the suggestions on user physical activity and eating behavior, and user perceptions of the automatically generated suggestions. Methods MyBehavior was designed to (1) use a combination of automatic and manual logging to track physical activity (eg, walking, running, gym), user location, and food, (2) automatically analyze activity and food logs to identify frequent and nonfrequent behaviors, and (3) use a standard machine-learning, decision-making algorithm, called multi-armed bandit (MAB), to generate personalized suggestions that ask users to either continue, avoid, or make small changes to existing behaviors to help users reach behavioral goals. We enrolled 17 participants, all motivated to self-monitor and improve their fitness, in a pilot study of MyBehavior. In a randomized two-group trial, investigators randomly assigned participants to receive either MyBehavior’s personalized suggestions (n=9) or nonpersonalized suggestions (n=8), created by professionals, from a mobile phone app over 3 weeks. Daily activity level and dietary intake was monitored from logged data. At the end of the study, an in-person survey was conducted that asked users to subjectively rate their intention to follow MyBehavior suggestions. Results In qualitative daily diary, interview, and survey data, users reported MyBehavior suggestions to be highly actionable and stated that they intended to follow the suggestions. MyBehavior users walked significantly more than the control group over the 3 weeks of the study (P=.05). Although some MyBehavior users chose lower-calorie foods, the between-group difference was not significant (P=.15). In a poststudy survey, users rated MyBehavior’s personalized suggestions more positively than the nonpersonalized, generic suggestions created by professionals (P<.001). Conclusions MyBehavior is a simple-to-use mobile phone app with preliminary evidence of efficacy. To the best of our knowledge, MyBehavior represents the first attempt to create personalized, contextualized, actionable suggestions automatically from self-tracked information (ie, manual food logging and automatic tracking of activity). Lessons learned about the difficulty of manual logging and usability concerns, as well as future directions, are discussed. Trial Registration ClinicalTrials.gov NCT02359981; https://clinicaltrials.gov/ct2/show/NCT02359981 (Archived by WebCite at http://www.webcitation.org/6YCeoN8nv).

MyBehavior: automatic personalized health feedback from user behaviors and preferences using smartphones

Mobile sensing systems have made significant advances in tracking human behavior. However, the development of personalized mobile health feedback systems is still in its infancy. This paper introduces MyBehavior, a smartphone application that takes a novel approach to generate deeply personalized health feedback. It combines state-of-the-art behavior tracking with algorithms that are used in recommendation systems. MyBehavior automatically learns a users physical activity and dietary behavior and strategically suggests changes to those behaviors for a healthier lifestyle. The system uses a sequential decision making algorithm, Multi-armed Bandit, to generate suggestions that maximize calorie loss and are easy for the user to adopt. In addition, the system takes into account users preferences to encourage adoption using the pareto-frontier algorithm. In a 14-week study, results show statistically significant increases in physical activity and decreases in food calorie when using MyBehavior compared to a control condition.

Objective Measurement of Sociability and Activity: Mobile Sensing in the Community

PURPOSE Automated systems able to infer detailed measures of a person’s social interactions and physical activities in their natural environments could lead to better understanding of factors influencing well-being. We assessed the feasibility of a wireless mobile device in measuring sociability and physical activity in older adults, and compared results with those of traditional questionnaires. METHODS This pilot observational study was conducted among a convenience sample of 8 men and women aged 65 years or older in a continuing care retirement community. Participants wore a waist-mounted device containing sensors that continuously capture data pertaining to behavior and environment (accelerometer, microphone, barometer, and sensors for temperature, humidity, and light). The sensors measured time spent walking level, up or down an elevation, and stationary (sitting or standing), and time spent speaking with 1 or more other people. The participants also completed 4 questionnaires: the 36-Item Short Form Health Survey (SF-36), the Yale Physical Activity Survey (YPAS), the Center for Epidemiologic Studies–Depression (CES-D) scale, and the Friendship Scale. RESULTS Men spent 21.3% of their time walking and 64.4% stationary. Women spent 20.7% of their time walking and 62.0% stationary. Sensed physical activity was correlated with aggregate YPAS scores (r2=0.79, P=.02). Sensed time speaking was positively correlated with the mental component score of the SF-36 (r2=0.86, P = .03), and social interaction as assessed with the Friendship Scale (r2=0.97, P = .002), and showed a trend toward association with CES-D score (r2=−0.75, P = .08). In adjusted models, sensed time speaking was associated with SF-36 mental component score (P = .08), social interaction measured with the Friendship Scale (P = .045), and CES-D score (P=.04). CONCLUSIONS Mobile sensing of sociability and activity is well correlated with traditional measures and less prone to biases associated with questionnaires that rely on recall. Using mobile devices to collect data from and monitor older adult patients has the potential to improve detection of changes in their health.

Leveraging Multi-Modal Sensing for Mobile Health: A Case Review in Chronic Pain

Active and passive mobile sensing has garnered much attention in recent years. In this paper, we focus on chronic pain measurement and management as a case application to exemplify the state of the art. We present a consolidated discussion on the leveraging of various sensing modalities along with modular server-side and on-device architectures required for this task. Modalities included are: activity monitoring from accelerometry and location sensing, audio analysis of speech, image processing for facial expressions as well as modern methods for effective patient self-reporting. We review examples that deliver actionable information to clinicians and patients while addressing privacy, usability, and computational constraints. We also discuss open challenges in the higher level inferencing of patient state and effective feedback with potential directions to address them. The methods and challenges presented here are also generalizable and relevant to a broad range of other applications in mobile sensing.

SARA: a mobile app to engage users in health data collection

Despite the recent progress in sensor technologies, many relevant health data can be only captured with manual input (e.g., food intake, stress appraisal, subjective emotion, substance use). A common problem of manual logging is that users often disengage within a short time because of high burden. In this work, we propose SARA, a novel app to engage users with ongoing tracking using timely rewards thereby reinforcing users for data input. SARA is developed for adolescents and emerging adults at risk for substance abuse. The rewards in SARA are designed to be developmentally and culturally appropriate to the target demographic and are theoretically grounded in the behavioral science literature. In this paper, we describe SARA and its rewards to increase data collection. We also briefly discuss future plans to evaluate SARA and develop just in time adaptive interventions for engagement and behavior change.

An intelligent crowd-worker selection approach for reliable content labeling of food images

Food journaling is an effective way to regulate excessive food intake. However manual food journaling is burdensome, and crowd-assisted food journaling has been explored to ease user burden. The crowd-assisted journaling uses a label & verify approach where an end-user uploads his/her food image and paid crowd-workers label content of the image. Then another set of crowd-workers verify the labels for correctness. In this paper, we propose an alternative approach where we label food images with only high performing labelers. Since high performing labelers generally provide good quality labels, our approach achieves high accuracy without verifying the food labels for correctness. We also propose a machine learning algorithm to automatically identify high performing crowd-labelers from a dataset of 3925 images collected over 5 months. Such automated identification of high performing workers and elimination of needless verification reduce cost of food labeling. Specially for large scale deployments where large number of images need to be labeled, our approach can reduce overall expenses by half.

Towards Health Recommendation Systems: An Approach for Providing Automated Personalized Health Feedback from Mobile Data

Personal data acquisition using smartphones has become robust and achievable in recent times: improvements in user interfaces have made manual inputting more straightforward and intuitive, while advances in sensing technology has made tracking more accurate and less obtrusive. Moreover, algorithmic advances in data mining and machine learning has led to better a interpretation and determination factors indicative of health conditions and outcomes. However, these indicators are still under-utilized when providing feedback to the user or a health worker. Mobile health systems that can exploit such indicators could potentially deliver precision feedback personalized to the user’s condition and also lead to increases in adherence and improve efficacy. In this book chapter, we will provide an overview of the state of the art in mobile health feedback systems and then discuss MyBehavior, an example of a feedback system that utilizes individual data streams and indicators. MyBehavior is the first personalized system that provides health beneficial recommendations based on physical activity and dietary data acquired using smartphones. The system learns common healthy and unhealthy behaviors from activity and dietary logs, and then prioritizes and suggests actions similar to existing behaviors. Such prioritization is done to promote a sense of familiarity to the suggestions and increase the likelihood of adoption. We also formulate a basis framework for future systems similar to MyBehavior and discuss challenges with regard to transference and adaptation.

Automated mobile systems for multidimensional well-being sensing and feedback

In recent years, we have seen a prolific rise of mobile and wearable sensing in healthcare and fitness. Although the data generated is incredibly useful, state-of-the-art feedback technologies are often limited to either providing an overall status or serving large volume of multi-dimensional sensor data with little processing. My research falls into filling this gap. I work on developing systems that use sensors to understand different dimensions of well being, and subsequently devise interventions through personalized and actionable suggestions. Using simple machine learning techniques, my systems automatically mine user behaviors that influence specific well-being dimensions. Then utilizing decision theory and behavioral psychology theory, my systems create personalized actionable suggestions that are related to existing users behaviors. In this proposal, I describe how I realize such automated systems for sensing and providing feedback.