Anxiety Detection Leveraging Mobile Passive Sensing
Lionel Levine, Migyeong Gwak, Kimmo Karkkainen, Shayan Fazeli, Bita Zadeh, Tara Peris, Alexander Young, Majid Sarrafzadeh
AAnxiety Detection Leveraging Mobile PassiveSensing
Lionel M. Levine
UCLA
Los Angeles, [email protected]
Migyeong Gwak
UCLA
Los Angeles, [email protected]
Kimmo K¨arkk¨ainen
UCLA
Los Angeles, [email protected]
Shayan Fazeli
UCLA
Los Angeles, [email protected]
Bita Zadeh
Chapman University
Orange County, [email protected]
Tara Peris
UCLA
Los Angeles, [email protected]
Alexander S. Young
UCLA
Los Angeles, [email protected]
Majid Sarrafzadeh
UCLA
Los Angeles, [email protected]
Abstract —Anxiety disorders are the most common class ofpsychiatric problems affecting both children and adults. However,tools to effectively monitor and manage anxiety are lacking, andcomparatively limited research has been applied to addressingthe unique challenges around anxiety. Leveraging passive andunobtrusive data collection from smartphones could be a viablealternative to classical methods, allowing for real-time mentalhealth surveillance and disease management. This paper presentseWellness, an experimental mobile application designed to tracka full-suite of sensor and user-log data off an individual’s devicein a continuous and passive manner. We report on an initialpilot study tracking ten people over the course of a month thatshowed a nearly 76% success rate at predicting daily anxietyand depression levels based solely on the passively monitoredfeatures.
Index Terms —mobile application, depression, remote mentalhealth monitoring, passive sensing, machine learning.
I. B
ACKGROUND AND I NTRODUCTION
Within the spectrum of mental health disorders, Anxietydisorders are the most common class of psychiatric problemsaffecting both children and adults [1] [2] [3], with up to onein three people in the US meeting full diagnostic criteria byearly adulthood [4] [5]. This breaks down roughly to 7 to9% suffering from a specific phobia, 7% from social anxietydisorder, and 2 to 3% each from panic disorder, agoraphobia,generalized anxiety disorder, and separation anxiety disorder.[6] Individuals with anxiety disorders contend with substantialdistress and impairment. They are at heightened risk for a hostof negative long-term outcomes including depression, sub-stance use, educational underachievement, and poor physicalhealth [7] [8] [9]The optimal method for the prevention or care of mentalillness is early identification, diagnosis, and proactive treat-ment [10]. Time-sensitive intervention is crucial for preventingconditions from becoming chronic and debilitating. However,traditional methods of psychiatric assessment, including clin-ical interviews and self-reports, are limited in their ability toprovide just-in-time intervention as well as early identification.They depend heavily on retrospective summaries collected in clinical settings, conditions that often result in reporting biases,inaccurate recall, or late and ineffectual treatment.Additionally, anxiety disorders are, for the most part, vastlyoverlooked and under-treated in the community; only 15-30%of anxious individuals in the community receive treatmentof any kind. Recent research has found strikingly high lev-els of anxiety among college-age youth. Indeed, 58.4% ofcollege-aged youth report feeling overwhelmed by anxiety[11]. Several other recent studies document the high proportionof college students meeting full diagnostic criteria for ananxiety disorder [12]. At the same time, young adults areparticularly overlooked within the health care system, withrates of screening, identification, and referral falling belowthose of either children or adults [9]. Given this landscape,there remains a pressing need for tools that improve earlyidentification of anxiety symptoms, provide users with thetools to monitor their activities, raise awareness of factorsimpacting on their wellbeing, and provide a mechanism forintervention should an anxiety episode escalate.The growing ubiquity of consumer devices, among themsmartphones, smartwatches, and in-home sensors, all equippedwith an array of sensors and user-logs, have resulted inan unprecedented opportunity to catalog and quantify thedaily aspects of an individuals life, creating repositories ofpersonalized information [13].While much has been noted about the insidious aspects ofsuch surveillance capabilities, there is also significant potentialfor such monitoring, if harnessed and utilized by the indi-viduals themselves, to dramatically improve their healthcareoutcomes. Such tools could potentially allow the user toaccurately track their behaviors and habits, compare personalactivities with population-level baselines, establish outlier be-haviors with their peers, and even motivate behavioral changeand the promotion of healthy habits.There is significant potential for such monitoring if har-nessed and utilized by the individuals themselves, to improvetheir healthcare outcomes dramatically. While physical behav-ior and physiological health are extensively tracked, mental a r X i v : . [ c s . C Y ] A ug ealth is largely overlooked.Specifically, The capability to track behavioral metrics andassociate them to mental health, although intimately linked,has not been definitively established. This owes to the sig-nificant difficulty in correlating monitorable behaviors andcorresponding mental health. Behavioral patterns both within(e.g., the transition from weekday to weekend) and acrossindividuals (e.g., simple differences in how many men andwomen carry their phones) are simply too diverse and toosubject to confounding factors beyond mental health to allowfor easy correlations. Nevertheless, the growing challengesaround mental health, necessitate exploring the possibilityfurther.Recent efforts have explored whether pervasive mentalhealth monitoring could be feasible through a smartphoneand the embedded sensors, such as motion sensors, ambientlight, microphone, camera, Global Positioning System (GPS),proximity, and touch screen [14] [15] [16] [17]. These effortshave shown the promise of this approach in successfullytying behavioral monitoring to mental health; however, suchapproaches have not translated into fully mature frameworks,and have focused almost exclusively on depression-relatedconditions, which while often spoken in conjunction withanxiety, manifest in distinct ways [18].The advantages of leveraging a smartphone-based platformare that the continuous and quantitative collection of datapotentially provides a more reliable indicator of an individual’srisk at any given time, as well as offering a mechanism forjust in time intervention should a mental health episode oc-cur [16]. Conversely, smartphone-derived data present severalchallenges, some of which have already been noted, which canresult in limited accuracy owing to differences in behavioralpatterns across users, and the indirect manner of detection [18].We present a system for the remote monitoring of mentalhealth symptoms, their fluctuation, and their attendant disrup-tion to personal functioning, called eWellness. The eWellnessframework is designed to capture a broad spectrum of remotemonitoring, survey data acquisition, secure data transmissionand management, data analytics, and visualization.The primary component of eWellness is a mobile appli-cation that facilitates data collection and transmission har-vested from an array of sensors and usage logs from a userssmartphone. The data is collected passively, pre-processed, andtransmitted through a secure gateway to the cloud, where it issecurely stored, and indexed using a scalable database.Concurrently the eWellness application includes an ac-tive querying component where users can be prompted withEcological Momentary Assessments (EMA) of their mentalhealth status. This architecture is complemented by a back-end analytic engine, capable of mapping observed metrics andexogenous data sources to a users mental health state, basedon adaptive statistical models, and advanced machine learningalgorithms. The system is designed to monitor overall mentalhealth as well as acute crisis events in both a retrospective andpredictive capacity. II. F RAMEWORK
A. Server
Data from the study, both usage-logs and EMAs response,are first encrypted and cached locally on the user’s device, andthen transmitted to a secure remote server, where it is storedin an encrypted scalable MySQL database.
B. eWellness Data Collection
The eWellness mobile application, developed for androiddevices, collects passive behavioral data derived from user-logs and embedded sensors, capturing the following metrics: • Communication : is monitored by incoming and outgoingphone calls and text messages, including the durationof phone calls, the number of texts and phone calls,and unique individuals contacted. This does not assessthe content of communications or the recipient of thecommunication, beyond establishing a unique contact. • Location : is periodically sampled using GPS, network,and Wi-Fi detection. Prompts for a new location aftermoving 5 meters, up to once a minute. This metric lever-ages the Google Fused Location API. The applicationdoes not track specific locations; instead, it keeps a totaldistance traveled using the vectorized haversine distancefunction. • Ambient Sound : detects speech and communicationabove 50 decibels using the phones microphone. It mon-itors every 5 minutes for 5 seconds. This metric does notassess the content of communications and merely recordsthe sound frequency and decibel level in a numeric value. • Activity and Movements : leverage the devices ac-celerometer, gyroscope, and GPS tracking. Activity issampled every 60 seconds. In order to determine station-ary and moving activity-type, the application leveragesGoogles Activity Recognition API. • Light : detects light level associated with possibly beingin an outdoor or indoor location. This sensor is sampledevery 6 seconds. • Phone use : is user-log monitoring the device’s screenon-time.We derived daily aggregated features from these metrics todefine the relationship between smartphone sensors and anx-iety symptom severity. We obtained statistical characteristics,such as minimum, maximum, mean, standard deviation, the25th, 50th, and 75th percentiles, of the numeric values of noiseexposure and the ambient luminance. The number of activitytransitions and duration of each physical activity per day alsobecame a significant metric of identifying mentally distresseddays.
C. Limiting Personally Identifiable Data Collection
Recognizing the potentially invasive nature of applicationslike this, data collection was carefully scoped to avoid thecollection of Personally Identifiable Information (PII) thatcould link a particular user to a particular dataset. For example,when attempting to gauge sociability, the application logs theotal number of phone calls made, total time on the phone,and the number of unique contacts called; the identities ofspecific callers were not tracked. This has the consequenceof introducing a degree of obscurity into an observed finding(e.g., as the application is unable to differentiate between callsto friends and calls to a customer-service hotline). At the sametime, in the interest of both respecting privacy and ensuring theacceptability of the app, these efforts were felt to be necessaryconstraints on data collection.III. P
ILOT S TUDY M ETHODOLOGYFig. 1. Screenshots of eWellness
An IRB-approved pilot study was conducted on a dozenindividuals who are using the Android version . and abovefrom the university community, including students and staff.Study participants did not have a reported history of mentalillness. Participants were asked to download and install theeWellness application (Figure 1), and then run it on theirphone for a month. Passive data was collected continuouslyby the application throughout the month. Participants wereasked to answer EMA daily through the eWellness app, butdid not provide any other personal information, such as name,gender, age, during participation.The Kessler Psychological Distress Scale (K10) [19] is avalidated measure of psychological distress during the past 30days, which is used for clinical and epidemiological purposes.It has a notable success in measuring feelings of anxiety alongwith depression. For this pilot, the K10 was modified to assesscriteria over the previous 24 hour period. The modified K10prompted the users as daily EMA to measure their feelingsof anxiety and depression. The K10 has ten items, which arescored from five through to one (all of the time, most of thetime, some of the time, a little of the time, and none of thetime). The minimum possible score of K10 is , and themaximum possible score is . K10 results are categorized intofour levels of psychological distress: low distress, moderatedistress, high distress, and very high distress (Table I). These TABLE IC
ATEGORIZATION OF
K10 S
CORES [20]K10 Score Level Samples (N=146)10-15 Low distress 9116-21 Moderate distress 2922-29 High distress 2130-50 Very high distress 5Fig. 2. 3-class (Low, Moderate, and High distress) Classification ConfusionMatrix. results were leveraged as a label for the classification ofsupervised learning. IV. R
ESULTS
Only 10 participants answered at least seven days of EMAsand provided successful passive sensing data throughout themonth. Our analysis focused on a fully supervised learningapproach, and only labeled samples were included. For thispilot study, we used 146 daily samples to identify daily anxietyand depression levels. The Z-Score normalization was appliedto the features to reach normalized values from differentparticipants.We selected 25 features that have a relatively higher corre-lation with the raw K10 score. For the 4-class classification,we used 5-fold Cross-Validation (CV) with four models: K-Nearest Neighbors (KNN), Extra-Trees (ET), Support VectorMachine (SVM), and Multilayer Perceptron (MLP). The classweight was automatically applied to the models inverselyproportional to the class frequencies to train the imbalanceddataset. The highest classification accuracy achieved wasaround 76% with the extra-trees model. We also applied theunder-sampling technique to improve the performance of animbalanced dataset. Samples from the low distress class wereremoved randomly to make uniformly distributed class labels.Samples from the very high distress class were also ignored.A confusion-matrix demonstrates that the average score ofclassifying three classes is . .V. D ISCUSSION
A. Limitations of the Study
While 10 subjects completing one-months worth of contin-uous data represents a critical validation of the technology andts potential utility, the dataset is too small to achieve statisti-cally significant results. Additionally, this pilot was scopedto only include individuals without a clinical diagnosis ofAnxiety or Depression. Consequently, there were insufficientcases of user-reported mental distress, particularly moderateor severe cases, to classify effectively. Additional studiesare planned to enlarge our dataset and include a cohort ofindividuals with diagnosed mental health conditions.There is also significant concern about the veracity of userself-reported labeling of mental health. The experimental de-sign, prompting users to fill out a daily EMA in the applicationvia push-notification, encouraged active participation in thestudy; however, there was no mechanism for confirming thatthe resulting inputs were an accurate reflection of a user’s ac-tual wellbeing. Users were likely motivated to respond quickly,and not necessarily accurately, which likely resulted in defaultanswers of no reported anxiety or depression. Furthermore,there may have been a reluctance among users to accuratelyreport out mental health issues given perceived embarrassmentor stigma associated with poor mental health. The authorsrecommend that future studies will have to address theseconcerns by better anticipating and correcting for challengeswith accurate labeling of mental health.Given the significant heterogeneity across subjects in-termsof usage-patterns, it was assumed that primary-success wouldbe achieved by classifying within users across time, ratherthan across users. The limitations of this initial dataset did notallow for adequate classifying by individual; however, the factthat classification success was achieved by bundling samplesacross all subjects is remarkable in its indication that cross-subject learning in this domain could be possible. Part of thisresult likely stems from normalization performed on the datato account for habitual differences in subject usage. Additionaldata collection is necessary to validate this finding.
B. Usability
Attempting to gauge the viability of the concept, participantsin the pilot were asked to submit a voluntary anonymizedpost-study questionnaire regarding their perceptions about theapplication and its data collection practices. All participantsresponded. A significant majority described the application assomewhat (40%) or mostly (40%) useful. Likewise, all usersendorsed feeling comfortable with the application, and onlyone user expressed reservations about the data being collected.All participants obtained detailed accounting of the data thatwas collected as part of their onboarding process to the study.No individual declined to participate after learning the precisenature of what was being tracked. This sampling suggests that,particularly among the young adults who are more accustomedto digitized lives, there is less concern about data collectionthrough their mobile devices. Limiting the collection of PIIcould be sufficient to assuage most privacy concerns.The primary issue users had with the application was itsbattery consumption resulting from heavy over-sampling ofthe sensors. Future iterations of the application will seek tooptimize battery usage by minimizing the sampling frequency. VI. C
ONCLUSION
Remote health monitoring of mental health, when done soleveraging passive and unobtrusive data collection, could bea useful alternative for conducting real-time mental healthsurveillance. This paper presents eWellness, an experimentalmobile application designed to track a full-suite of sensor andlog data off a user’s device continuously and passively. Aninitial pilot study tracking ten people over a month showeda nearly 76% success rate at predicting daily anxiety levelsbased solely on the passively monitored features. Our currentapproach may prove useful at tracking longitudinal trends inan individual’s mental health. Additional work is needed torefine both the technology and analytics.R
EFERENCES[1] R. H. Bitsko, J. R. Holbrook, R. M. Ghandour, S. J. Blumberg, S. N.Visser, R. Perou, and J. T. Walkup, “Epidemiology and impact of healthcare provider–diagnosed anxiety and depression among us children,”
Journal of developmental and behavioral pediatrics: JDBP , vol. 39,no. 5, p. 395, 2018.[2] S. Cartwright-Hatton, “Anxiety of childhood and adolescence: Chal-lenges and opportunities,” 2006.[3] K. R. Merikangas, J.-p. He, M. Burstein, S. A. Swanson, S. Avenevoli,L. Cui, C. Benjet, K. Georgiades, and J. Swendsen, “Lifetime preva-lence of mental disorders in us adolescents: results from the nationalcomorbidity survey replication–adolescent supplement (ncs-a),”
Journalof the American Academy of Child & Adolescent Psychiatry , vol. 49,no. 10, pp. 980–989, 2010.[4] M. Topper, P. M. Emmelkamp, E. Watkins, and T. Ehring, “Preventionof anxiety disorders and depression by targeting excessive worry andrumination in adolescents and young adults: a randomized controlledtrial,”
Behaviour research and therapy , vol. 90, pp. 123–136, 2017.[5] P. Hammerness, T. Harpold, C. Petty, C. Menard, C. Zar-Kessler, andJ. Biederman, “Characterizing non-ocd anxiety disorders in psychiatri-cally referred children and adolescents,”
Journal of affective disorders
Journal of AffectiveDisorders , vol. 165, pp. 45–52, 2014.[8] A. M. Bardone, T. E. Moffitt, A. Caspi, N. Dickson, W. R. Stanton,and P. A. Silva, “Adult physical health outcomes of adolescent girlswith conduct disorder, depression, and anxiety,”
Journal of the AmericanAcademy of Child & Adolescent Psychiatry , vol. 37, no. 6, pp. 594–601,1998.[9] L. J. Woodward and D. M. Fergusson, “Life course outcomes of youngpeople with anxiety disorders in adolescence,”
Journal of the AmericanAcademy of Child & Adolescent Psychiatry , vol. 40, no. 9, pp. 1086–1093, 2001.[10] S. Wagner, C. Koehn, M. White, H. Harder, I. Schultz, K. Williams-Whitt, O. Warje, C. Dionne, M. Koehoorn, R. Pasca et al. , “Mentalhealth interventions in the workplace and work outcomes: a best-evidence synthesis of systematic reviews,”
Int J Occup Environ Med(The IJOEM) , vol. 7, no. 1 January, pp. 607–1, 2016.[11] A. C. H. Association et al. , “American college health association–national health assessment ii: Reference group executive summary spring2016,” 2016.[12] R. Bruffaerts, P. Mortier, G. Kiekens, R. P. Auerbach, P. Cuijpers, K. De-myttenaere, J. G. Green, M. K. Nock, and R. C. Kessler, “Mental healthproblems in college freshmen: Prevalence and academic functioning,”
Journal of affective disorders , vol. 225, pp. 97–103, 2018.[13] M. Swan, “Health 2050: The realization of personalized medicinethrough crowdsourcing, the quantified self, and the participatory bioc-itizen,”
Journal of personalized medicine , vol. 2, no. 3, pp. 93–118,2012.14] L. Ciabattoni, F. Ferracuti, S. Longhi, L. Pepa, L. Romeo, and F. Verdini,“Real-time mental stress detection based on smartwatch,” in . IEEE,2017, pp. 110–111.[15] J. Osma, I. Plaza, E. Crespo, C. Medrano, and R. Serrano, “Proposalof use of smartphones to evaluate and diagnose depression and anxietysymptoms during pregnancy and after birth,” in
IEEE-EMBS Interna-tional Conference on Biomedical and Health Informatics (BHI) . IEEE,2014, pp. 547–550.[16] D. Ben-Zeev, E. A. Scherer, R. Wang, H. Xie, and A. T. Campbell,“Next-generation psychiatric assessment: Using smartphone sensors tomonitor behavior and mental health.”
Psychiatric rehabilitation journal ,vol. 38, no. 3, p. 218, 2015.[17] N. Narziev, H. Goh, K. Toshnazarov, S. A. Lee, K.-M. Chung, andY. Noh, “Stdd: Short-term depression detection with passive sensing,”
Sensors , vol. 20, no. 5, p. 1396, 2020.[18] Y. Fukazawa, T. Ito, T. Okimura, Y. Yamashita, T. Maeda, and J. Ota,“Predicting anxiety state using smartphone-based passive sensing,”
Jour-nal of biomedical informatics , vol. 93, p. 103151, 2019.[19] G. Andrews and T. Slade, “Interpreting scores on the kessler psycholog-ical distress scale (k10),”