Akane Sano

Stress Recognition Using Wearable Sensors and Mobile Phones

In this study, we aim to find physiological or behavioral markers for stress. We collected 5 days of data for 18 participants: a wrist sensor (accelerometer and skin conductance), mobile phone usage (call, short message service, location and screen on/off) and surveys (stress, mood, sleep, tiredness, general health, alcohol or caffeinated beverage intake and electronics usage). We applied correlation analysis to find statistically significant features associated with stress and used machine learning to classify whether the participants were stressed or not. In comparison to a baseline 87.5% accuracy using the surveys, our results showed over 75% accuracy in a binary classification using screen on, mobility, call or activity level information (some showed higher accuracy than the baseline). The correlation analysis showed that the higher-reported stress level was related to activity level, SMS and screen on/off patterns.

Recognizing academic performance, sleep quality, stress level, and mental health using personality traits, wearable sensors and mobile phones

What can wearable sensors and usage of smart phones tell us about academic performance, self-reported sleep quality, stress and mental health condition? To answer this question, we collected extensive subjective and objective data using mobile phones, surveys, and wearable sensors worn day and night from 66 participants, for 30 days each, totaling 1,980 days of data. We analyzed daily and monthly behavioral and physiological patterns and identified factors that affect academic performance (GPA), Pittsburg Sleep Quality Index (PSQI) score, perceived stress scale (PSS), and mental health composite score (MCS) from SF-12, using these month-long data. We also examined how accurately the collected data classified the participants into groups of high/low GPA, good/poor sleep quality, high/low self-reported stress, high/low MCS using feature selection and machine learning techniques. We found associations among PSQI, PSS, MCS, and GPA and personality types. Classification accuracies using the objective data from wearable sensors and mobile phones ranged from 67-92%.

Toward a taxonomy of autonomic sleep patterns with electrodermal activity

This paper presents a first version of a taxonomy of automatic sleep patterns found with the Affectiva Q™ Sensor, a wireless, logging biosensor that measures skin conductance, skin temperature, and motion comfortably from the wrist. Several studies have examined electrodermal activity (EDA) during sleep, but they focused on an analysis of EDA for only a small number of nights. We quantitatively analyzed EDA during sleep in three study situations: (1) Comparing EDA with polysomnography (PSG) from seven subjects in a sleep lab, (2) Characterizing multiple nights of EDA in a sleep lab, in a hospital and at home from 24 subjects, and (3) Gathering long-term EDA (30–60 nights) patterns from three subjects during home sleep. After gathering this rich corpus of data, we characterized inter- and intra-individual differences of EDA features and the relation of EDA peaks to subjective sleep quality. Here we present results from the three studies in an effort to begin to characterize autonomic patterns found in natural sleep.

Email Duration, Batching and Self-interruption: Patterns of Email Use on Productivity and Stress

While email provides numerous benefits in the workplace, it is unclear how patterns of email use might affect key workplace indicators of productivity and stress. We investigate how three email use patterns: duration, interruption habit, and batching, relate to perceived workplace productivity and stress. We tracked email usage with computer logging, biosensors and daily surveys for 40 information workers in their in situ workplace environments for 12 workdays. We found that the longer daily time spent on email, the lower was perceived productivity and the higher the measured stress. People who primarily check email through self-interruptions report higher productivity with longer email duration compared to those who rely on notifications. Batching email is associated with higher rated productivity with longer email duration, but despite widespread claims, we found no evidence that batching email leads to lower stress. We discuss the implications of our results for improving organizational email practices.

Automatic identification of artifacts in electrodermal activity data

Recently, wearable devices have allowed for long term, ambulatory measurement of electrodermal activity (EDA). Despite the fact that ambulatory recording can be noisy, and recording artifacts can easily be mistaken for a physiological response during analysis, to date there is no automatic method for detecting artifacts. This paper describes the development of a machine learning algorithm for automatically detecting EDA artifacts, and provides an empirical evaluation of classification performance. We have encoded our results into a freely available web-based tool for artifact and peak detection.

Multimodal annotation tool for challenging behaviors in people with Autism spectrum disorders

Individuals diagnosed with Autism Spectrum Disorders (ASD) often have challenging behaviors (CBs), such as self-injury or emotional outbursts, which can negatively impact the quality of life of themselves and those around them. Recent advances in mobile and ubiquitous technologies provide an opportunity to efficiently and accurately capture important information preceding and associated with these CBs. The ability to obtain this type of data will help with both intervention and behavioral phenotyping efforts. Through collaboration with behavioral scientists and therapists, we identified relevant design requirements and created an easy-to-use mobile application for collecting, labeling, and sharing in-situ behavior data in individuals diagnosed with ASD. Furthermore, we have released the application to the community as an open-source project so it can be validated and extended by other researchers.

MOVEMENT-RELATED CORTICAL EVOKED POTENTIALS USING FOUR-LIMB IMAGERY

We compared the electroencephalographic changes during actual and imaginary movements with four limbs and classified optimally the responses during four-limb imagery. Evoked potentials in imagery exhibited lower and delayed peaks compared to actual-movement responses, but activations in the primary and the supplementary motor area were similar. Source-modeling analysis revealed that the motor and the parietal cortex were activated similarly, but several dipole sources were active in the frontal cortex for imagery. We compared thirteen classification methods and a combination of template matching and time-frequency methods showed the highest average of 70% classification rate for all limbs.

Comparison of sleep-wake classification using electroencephalogram and wrist-worn multi-modal sensor data

This paper presents the comparison of sleep-wake classification using electroencephalogram (EEG) and multi-modal data from a wrist wearable sensor. We collected physiological data while participants were in bed: EEG, skin conductance (SC), skin temperature (ST), and acceleration (ACC) data, from 15 college students, computed the features and compared the intra-/inter-subject classification results. As results, EEG features showed 83% while features from a wrist wearable sensor showed 74% and the combination of ACC and ST played more important roles in sleep/wake classification.

Recognition of sleep dependent memory consolidation with multi-modal sensor data

This paper presents the possibility of recognizing sleep dependent memory consolidation using multi-modal sensor data. We collected visual discrimination task (VDT) performance before and after sleep at laboratory, hospital and home for N=24 participants while recording EEG (electroencepharogram), EDA (electrodermal activity) and ACC (accelerometer) or actigraphy data during sleep. We extracted features and applied machine learning techniques (discriminant analysis, support vector machine and k-nearest neighbor) from the sleep data to classify whether the participants showed improvement in the memory task. Our results showed 60–70% accuracy in a binary classification of task performance using EDA or EDA+ACC features, which provided an improvement over the more traditional use of sleep stages (the percentages of slow wave sleep (SWS) in the 1st quarter and rapid eye movement (REM) in the 4th quarter of the night) to predict VDT improvement.

Intelligent Mobile, Wearable, and Ambient Technologies for Behavioral Health Care

This chapter provides an overview of intelligent mobile, wearable, and ambient device applications for behavioral health care. Several of the latest advancements in these technologies are presented and descriptions of applicable artificial intelligence methods and technologies are provided. Examples of their practical applications in behavioral and mental health care are also provided. Design recommendations are given and relevant security, privacy and ethical considerations specific to the use of these technologies are discussed. The chapter concludes with a discussion of emerging technologies and opportunities.