MMental Health and Sensing
Abdul Kawsar Tushar , Muhammad Ashad Kabir , and Syed IshtiaqueAhmed Department of Computer Science, University of Toronto, Toronto, Canada School of Computing and Mathematics, Charles Sturt University, NSW, [email protected], [email protected], [email protected]
Abstract.
Mental health is a global epidemic, affecting close to half abillion people worldwide. Chronic shortage of resources hamper detec-tion and recovery of affected people. Effective sensing technologies canhelp fight the epidemic through early detection, prediction, and result-ing proper treatment. Existing and novel technologies for sensing mentalhealth state could address the aforementioned concerns by activatinggranular tracking of physiological, behavioral, and social signals pertain-ing to problems in mental health. Our paper focuses on the availablemethods of sensing mental health problems through direct and indirectmeasures. We see how active and passive sensing by technologies as wellas reporting from relevant sources can contribute toward these detectionmethods. We also see available methods of therapeutic treatment avail-able through digital means. We highlight a few key intervention tech-nologies that are being developed by researchers to fight against mentalillness issues.
Keywords:
Mental health, wearables, sensing.
Mental health can be termed as a concern that is plaguing the entire earth. Thereis a worrying number of 450 million around the globe that have been diagnosedwith some form of mental or neurodevelopmental illnesses [1] and they often leadto various levels of disability [2]. Mental and neurodevelopmental illnesses giverise to a mortality rate that has been compared at a level more than double thatof the general population, leading to approximately 8 million deaths [3]. Anotherimpact of such sheer numbers related to these conditions is the financial burden,which generate from expenditure for care as well as the loss in productivity. Thenumbers related to economic loss has been estimated at more than $400 billiondollars, that only in the United States of America for a year [4].Well-being of patients suffering from serious mental illness for a sustainableperiod can be ensured through treatment, management, and care and this canbe achieved through granular symptom monitoring [5]. However, existing clinicaltools and resources are limited in terms of accessibility and scalability [6]. Mo-bile health, often termed as mHealth for brevity, is where mobile (or electronic)devices converge with medical professionals and public health administration [7] a r X i v : . [ c s . H C ] S e p Mental Health and Sensing and has been a well-researched area for exploring the scope of involving qualita-tive research methods with a view to providing accessible treatment, participantmonitoring and retention, and progress of treatment. The growth in the numberof mobile devices has also been a significant factor in lending more weight to thistype of solutions, since close to 4 billion people around the world own at least onephone( the number is scheduled to double by 2022) [8]. This is remarkable whenwe consider the fact that studies found more than 70% people suffering fromserious mental illness have mobile phones [9]. In addition, an increase of sensorsembedded in mobile phones is giving birth to novel possibilities of utilizing thesedevices into mental health care based on digital evidence, such as quantitativefunctional and behavioral labels efficiently and without obstacles [10, 11].What is holding an widespread adoption of sensors in mental health careand management is the scattered and restricted state of evidence that provesthe connection between, on one hand, sensor data obtained using wearables andubiquitous smartphones and, on the other hand, the prevalence and status ofmental illnesses [6, 12]. In this paper, we show how technology can help in detec-tion and sensing of mental health problems around the around. Specifically, wefocus on the major mental illnesses that are tormenting billions of people acrossvarious countries. We see how active and passive sensing by technologies as wellas reporting from relevant sources can contribute toward these detection meth-ods. We also see available methods of therapeutic treatment available throughdigital means. We highlight a few key intervention technologies that are beingdeveloped by researchers to fight against mental illness issues.
In this section, we do not aim to classify mental health disorders as that isnot our target for this paper. Instead, our discussion would revolve around theprevalence of these disorder to provide a sense of their different presentations.Characteristics of major mental disorders include a permutation of irregular andatypical belief, concepts, attitude, and expression especially with people in thesurrounding. Top mental disorders are schizophrenia, bipolar conditions (BP),depression and sadness, and developmental disorders together with autism [13].Table 1 shows the number of people affected by these disorders in the world.
Prevalence:
We use the World Health Organization fact sheet [13] to knowthe prevalence of these illnesses. Depression is a common psychiatric disorderthat is a major concern responsible for various forms of disability. Around theworld, a population of more than 200 million are affected by depression, majorityof whom are women. Schizophrenia can make it difficult for people affected towork or study normally. More tan 40 million humans are affected by BP, whichcomprises of both manic and depressive periods mingled with regular behaviorand mood states. Schizophrenia is another crippling psychiatric disorder thatdisables around 15 million. Schizophrenia and other psychoses can distort theability to think, perceive, and express. Another concerning illness that affectsmore than 50 million worldwide is dementia, which is characterized by a dete- ental Health and Sensing 3
Mental HealthProblem Severity in theWorld
Depression 264 millionBipolar Disorder 45 millionSchizophrenia 20 millionDementia 50 millionAutism Spectrum Disorder 50 million
Table 1.
Major mental health disorders and victims of each of them in the world. rioration in cognitive function that exacerbates the regular effects of aging onmemory. Lastly, developmental disorders comprise various terms such as intel-lectual disability and pervasive developmental disorders including autism. 1 in160 children in the world are estimated to be affected by autism [13].
In this section, we discuss some of the widely popular methods that are used todetect the prevalence and existence of mental health problems in a person. Wefirst discuss how the wide use of mobile phone technology is helping the revolu-tion of mental health sensing methods. Then we discuss the currently availablesensing technologies by dividing these technologies in three categories: biologi-cal or physiological sensing, behavioral or soft sensing, and social or proximitysensing methods.
The wide availability of mobile and sensor technology has enabled the opportu-nity for personalized data collection efficiently and without obstacles.
Real-world behavior sensing:
A popular technique of estimating the sta-tus of one’s mental health is to use smartphones sensors to capture behavioraldata of humans. We can use a plethora of sensors available in today’s smart-phones. The aforementioned sensors can be used in different permutations tocapture a wide range of human behavior, including mental and physical. A pre-decessor to this approach was used along with various sensors to capture andclassify data of physical activity [14]. Another similar study was utilized to pre-dict social isolation in older adults using sensors and body microphone [15].
Technology-mediated behavior:
More and more work is exploring andanalyzing online behavior to predict and care for mental health. For instance, arecent study by de Choudhury et al. demonstrated using information from socialmedia to predict the early signs of depression [16].
Mental Health and Sensing Facial expression:
Facial expressions and associated emotions can conveya persons emotional states and could be useful in diagnosing psychiatricillnesses [17]. Tron et al. tracked facial Action Units using 3D cameras inpeople with schizophrenia to differentiate patients from control participants[18]. Another study used a similar range of technology to detect the onset ofsuicidal thoughts in a person [19]. There are other similar studies [20, 21].These ideas can be paired with the widespread use of mobile phones todiagnose mental illness.2.
Heart rate variability:
People affected with psychiatric conditions suf-fer from a high chance of facing cardiovascular morbidity compared to thegeneral population [22–24]. A recent research argued that the reduction inheart rate variability (HRV) could be behind the connection between thisheightened cardiac mortality and psychiatric illness [25]. If that is indeedtrue, then there is a link between HRV and depression and this could bedetected through sensing. The same relationship is reported for people withpost-traumatic stress disorder [26], anxiety disorders [27], and bipolar disor-der and schizophrenia [28]. Traditional HRV measurement devices are heavy;therefore, smart devices can be an apt replacement. Some devices are alreadyin place for this purpose [29].3.
Eye movement:
Data captured from fine changes in the movement of eyescan be used to draw conclusion and inferences about mental illness, as shownin research on schizophrenia [30] and depression [31, 32]. A technology calledEOG glasses (Electrooculography in Wearable form as in glasses) can be usedfor detecting the movement of eyes and well as the frequency and pattern ofblinking [33]. Additionally, web cameras can also be used for similar purposesto detect the beginning of dementia [34].4.
Electrodermal activity:
Electrodermal activity (or EDA for short) canbe gauged from measuring changes in electrical properties in human skin[35]. Various research has utilized this technique in mental health detectionsetting to utilize the proposed relationship between heightened EDA andmental illness [36]. For instance, regarding BP, EDA signals can classifyvarious mood states and subsequent swings [37, 38]. EDA can also be usedto determine suicidal tendencies [39, 40]. Mobility and location:
Patterns in our location can be used to predictsocial activities, which can provide a peek into a person’s mental status. Aresearch tried to link lethargic lifestyle with depression [41], while anotherlinked phone location data to depression severity [42]. There are various suchother researches (see [43, 44]).2.
Speech patterns:
Characteristics of speech could be used to judge mentalhealth. There is an established relation between depression and human voice[45], as well as speech monotone [46]. A recent research found that jitters in ental Health and Sensing 5 voice is important to establish a pattern that can identify patients havingsuicidal thoughts [47]. Three other research has done significant work indeveloping frameworks to collect audio data for similar purposes [48–50].3.
Technology use:
The patterns demonstrated in our day-to-day technologyusage contain meaningful data and this can be leveraged in the fight againstmental illnesses. For instance, patterns of phone use have been linked withpeople’s behaviors related to sleeping and waking up, something that hasbeen explored in relation to screen lock information of modern smartphonedevices [51]. These features were discovered to be significantly correlatedwith mood in BP [52].Other features pertaining to the pattern of usage of smartphone by a personhave been found helpful. Two very recent papers explored the connectionbetween change of technology usage and different mood states of BP patients[53, 54]. Schizophrenia patients also found to display similar connection [44],as did patients of depression [43].4.
Activity:
While the connection between physical activity and mental statemay not seem obvious, research has actually established a strong link be-tween them. For instance, in BP, mania and depression are strongly relatedwith overactivity and under-activity, respectively [55]. This information canbe utilized to detect and predict the onset, existence, and duration of eitherof these mood states in a person. Studies earlier used actigraph to determinelevel of activity in a person [56, 57]. However, different smartphone sensorscould be utilized in order to monitor an user activity level on a continuousbasis [58]. Studies in recent years have looked at data of numerous physicalactivities to connect them with mental disorders line BP [59, 60], depression[42], and schizophrenia [44]. Social interaction:
Sensing technologies that can capture data from nearbyaccess points as well as other sensing devices are helpful in obtaining infor-mation about social interactions without much obstructions. Although notas accurate as user-filled data, these social proximity data can be used as areplacement which can serve to indicate the level of social interactions of aperson with the help of a good algorithm [61, 62].2.
Communication patterns:
A person’s level of social engagement can beindicated by their communication technology usage data. Communicationpatterns of humans have been used to identify various mental illnesses such asBP [17] and schizophrenia [44, 63]. They found that symptoms are associatedwith a change of frequency and duration in outgoing SMS or phone calls.3.
Social media:
Since many individuals are comfortable in sharing their dailylife stories on different social media channels, these channels can be a richsource of data to determine social engagement, social network characteris-tics, mood, and emotion that are related to ones overall well-being. Recentresearch has worked with Twitter data to predict depression and PTSD
Mental Health and Sensing [16, 64]. Images and videos in social media posts can be used to collect use-ful signals about a person. For example, mood [65] and depression [66] canbe predicted from sentiment visible in social media photos. This idea has thepotential to be scalable in this age of vast social media user engagement.
In this section, we explore some of the available intervention techniques thatare associated with digital sensing of mental health problems. There are variousmethods that are used to assist people in judging and caring for their mentalhealth conditions. These technologies can range from using machine learningmethods to detect mental illness to providing customized and personalized feed-back or treatment routine. In the end, we discuss two popular methods thatseamlessly integrate the sensing, inference, and partial treatment procedures.If we want to a scientific model of how humans behave from the data that wecapture from various wearable and mobile sensing devices, we need to developone or more algorithms that will serve as a bridge between what we know andwhat we want to predict. This is useful for medical doctors as well as publichealth administrators alike to have useful insights about the spread and depthof mental illnesses. While statistical modeling can be used as for this purpose,they may not capture the high level of complexity present in human behavioraldata [67]. For this reason, researchers have been opting for machine learningmethods [68]. For instance, depression [42, 43] and social rhythm metric [69] hasbeen inferred using smartphone data that was passively acquired. Specifically,deep learning has shown success in determining or predicting about the onset ofmental illness [70].
Usually, there are two approaches for presenting the captured data using wear-able and mobile sensors to the user for useful insights and feedback in themHealth domain. First, we can summarize the collected data into short burstsof information or visual statistics or both (e.g., UbiFit [71] and BeWell [68]).Second, based on the collected data, we can predict a set of present and futurestates and based on this prediction, we can provide recommendations to the user.While the first method usually allows for the user to set their goal based on thepresented statistics, the second method would often set a goal for the user andprovide recommendation to reach that goal with minimum effort and maximumefficiency. The issue is that it is difficult to make maximum or even desired use ofthe complexity present in the capture data. Hence, the feedback or recommenda-tion presented to people are only partially useful and even sometimes partiallycorrect. Even more, because of the lack of insight into the data, the recommen-dations can even be dangerous sometimes. The good news is that researchershave been exploring different frameworks that would formalize the process ofproviding feedback and recommendation that would streamline the process and ental Health and Sensing 7
Fig. 1.
Example of a layered, hierarchical sensemaking framework used in [11]. Boxesin blue are high-level behavioral markers. Boxes in yellow are features. Boxes in greenare inputs to the sensing platform. minimize risks. Additionally, researchers have explore variation in feedback inthe forms of data change, augmentation, or subtraction to perceive the resultingchange in user behavior [72]. Below, we discuss two such frameworks.
A hierarchical framework extracts data from sensors and extract useful featuresin two sequential steps. This framework is depicted in Figure 1. The first levelat the bottom (in green) has the raw inputs to the sensor device which is often amobile phone. This data needs to be processed to extract any useful information.The second level (in yellow) is where the system merges statistical algorithms–expert in finding patterns in data– with human intelligence via brainstormingand relevant domain expertise to construct low-level features. The boxes at thetop (in blue) combine the middle-level features into behavioral markers by usingmachine learning and deep learning.Aung et al. [73] propose a more comprehensive and inclusive three-stageframework for integrating behavior sensing into the domain of mental health.This model can take into account user data from various physical sensors aswell as self-reported data. Figure 2 shows the entire process. The first stagetakes into account data from sensors as well as self-reported procedures over atime to become more inclusive. Then, this raw data is processed using machinelearning or other methods to create new features and gain useful insights into thedata. The final phase works on integrating the inferences from the middle steptoward the management of the condition if detected. This could include using theinferences alongside traditional mental health therapies to create personalizeddigital interventions and chrono-therapeutic interventions.
Mental Health and Sensing
Fig. 2.
Framework for using behavioral sensing for detection of psychiatric illnesshealth used in [73]
The existing technologies for sensing mental health condition suffer from a num-ber of limitations. First, studies do not demonstrate a significant level of ef-fectiveness for and correlation with improving patient detection and care formental health patients. The dearth of clinical evidence can be linked back to theinsufficient amount of research, lack of significant sample size or population, aninsignificant time over which studies were live (both in short term and mediumterm), as well as a lack of funding opportunities, which can again lead to a lackof awareness regarding the severity and spread of these problems in differentsocieties. These issues can be addressed by improving the study design and pro-cedures and seeking more funding [6]. Increasing awareness about mental healthproblems is another wing of solving these issues.Second, not many studies strive to combine the effects of various types ofsensing. If data is being collected about a single patient or the same group ofpatients and the result inferred from these sources are consistent, then the sys-tem could be more confident about its inference. The challenge of combiningdata from different streams is chiefly computational, where machine learningsystems could provide to be a helping hand. However, machine learning meth-ods also suffer from the lack of reproducibility [74]. These techniques used formental health sensing suffer from a lack of expiration date as well as the curseof variability [11]. Additionally, how the errors associated with the predictionsof a machine learning system will be explained, addressed, and incorporated inthe future iterations of the same model is something that needs more significantresearch, which is presently absent. Such user-facing errors need to be addressedproperly and with a convincing methodology, lest we should face affecting thequality of the experience of people using technology for sensing..Third, the nature of sensing technologies dictate that they will need to ob-tain a gigantic amount of data that is related to personal use and has a potentialto sensitive in nature (especially when the data is related to personal behavior ental Health and Sensing 9 or health problems). Furthermore, wearable sensing technologies are not ad-vanced enough to differentiate between target participant and non-participantsand hence, due to the availability and ease of use in any setting, can risk ob-taining unexpected, irrelevant, and unnecessary data [75] that can lead to falsediagnosis, false triggers, and lack of care for patients. Researchers should striveto identify potentially sensitive data items to deploy a plan of action of protect-ing this data from malicious parties. In addition, researchers should also thinkabout how they can identify unexpected, irrelevant, and unnecessary data andseparate valuable data from all the clutter.Fourth, related to the previous concern is another that is the issue of privacyand security for passively collected data. Unfortunately, the existing researchcommunity members are in a disagreement regarding what to do about thisrisk [76]. A crucial aspect that may often get overlooked is the difficulty instripping identifying information from data collected by sensors in mobile phoneand wearables. Even seeming innocuous data such as location information [77],when obtained in sufficient quality, can expose the identification of a victim,exposing to a risk of facing social stigma [78]. While there are numerous existingtechniques to help de-identify data (especially location data), they are not full-proof in preserving privacy as well as usefulness of data at the same time [79].While an exponentially increasing amount of data is being captured from mo-bile and wearable sensors around the world, there are other, less structured chal-lenges that can pose significant barriers toward capturing and treating mentalillness using mobile devices. In Global South, mental health is a vastly neglecteddomain rife with misconceptions, maltreatment, and lack of treatment options[80]. Sometimes, people consult the local witches instead of medical doctors totreat illnesses that seem unusual to the community (like mental illness) [81].Rural areas especially suffer from the lack of medical infrastructure and mentalhealth facilities [82] and if we plan to offset this restriction by employing mobiledevices, we face another different set of challenges [75, 83]. Furthermore, thelow rate of literacy is often a hurdle for people to use smartphones [84]. Thesechallenges should also be considered along with the more “high-tech” obstaclesdescribed above if we want to capture a holistic picture of the domain where wewant to treat and manage psychiatric illnesses with sensors. ibliography [1] Brundtland GH (2001) Mental health: new understanding, new hope. Jama286(19):2391–2391[2] Organization WH, et al (2003) Investing in mental health. WHO p 36[3] Walker ER, McGee RE, Druss BG (2015) Mortality in mental disorders andglobal disease burden implications: a systematic review and meta-analysis.JAMA psychiatry 72(4):334–341[4] Insel T (2015) Mental health awareness month: By the numbers. NationalInstitute of Mental Health blog[5] Morriss R, Vinjamuri I, Faizal MA, Bolton CA, McCarthy JP (2013) Train-ing to recognize the early signs of recurrence in schizophrenia. Schizophreniabulletin 39(2):255[6] Abdullah S, Choudhury T (2018) Sensing technologies for monitoring seri-ous mental illnesses. IEEE MultiMedia 25(1):61–75[7] Adibi S (2015) Mobile health: a technology road map, vol 5. Springer[8] Cerwall P, Jonsson P, M¨oller R, B¨avertoft S, Carson S, Godor I (2015) Eric-sson mobility report. On the Pulse of the Networked Society Hg v Ericsson[9] Ben-Zeev D, Davis KE, Kaiser S, Krzsos I, Drake RE (2013) Mobile tech-nologies among people with serious mental illness: opportunities for futureservices. Administration and Policy in Mental Health and Mental HealthServices Research 40(4):340–343[10] Gaggioli A, Riva G (2013) From mobile mental health to mobile wellbeing:opportunities and challenges. In: MMVR, pp 141–147[11] Mohr DC, Zhang M, Schueller SM (2017) Personal sensing: understandingmental health using ubiquitous sensors and machine learning. Annual reviewof clinical psychology 13:23–47[12] Or F, Torous J, Onnela JP (2017) High potential but limited evidence: Us-ing voice data from smartphones to monitor and diagnose mood disorders.Psychiatric rehabilitation journal 40(3):320[13] WHO (2019) Mental disorders by world health organization. , accessed:2019-12-24[14] Lester J, Choudhury T, Borriello G (2006) A practical approach to recogniz-ing physical activities. In: International conference on pervasive computing,Springer, pp 1–16[15] Rabbi M, Ali S, Choudhury T, Berke E (2011) Passive and in-situ assess-ment of mental and physical well-being using mobile sensors. In: Proceed-ings of the 13th international conference on Ubiquitous computing, ACM,pp 385–394[16] De Choudhury M, Gamon M, Counts S, Horvitz E (2013) Predicting de-pression via social media. In: Seventh international AAAI conference onweblogs and social media ental Health and Sensing 11 [17] Gur RE, Kohler CG, Ragland JD, Siegel SJ, Lesko K, Bilker WB, GurRC (2006) Flat affect in schizophrenia: relation to emotion processing andneurocognitive measures. Schizophrenia bulletin 32(2):279–287[18] Tron T, Peled A, Grinsphoon A, Weinshall D (2015) Automated facial ex-pressions analysis in schizophrenia: A continuous dynamic approach. In:International Symposium on Pervasive Computing Paradigms for MentalHealth, Springer, pp 72–81[19] Laksana E, Baltruˇsaitis T, Morency LP, Pestian JP (2017) Investigatingfacial behavior indicators of suicidal ideation. In: 2017 12th IEEE Interna-tional Conference on Automatic Face & Gesture Recognition (FG 2017),IEEE, pp 770–777[20] Valstar M, Schuller B, Smith K, Almaev T, Eyben F, Krajewski J, CowieR, Pantic M (2014) Avec 2014: 3d dimensional affect and depression recog-nition challenge. In: Proceedings of the 4th International Workshop on Au-dio/Visual Emotion Challenge, ACM, pp 3–10[21] Wang R, Campbell AT, Zhou X (2015) Using opportunistic face loggingfrom smartphone to infer mental health: challenges and future directions.In: Adjunct Proceedings of the 2015 ACM International Joint Conferenceon Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACMInternational Symposium on Wearable Computers, ACM, pp 683–692[22] Hennekens CH, Hennekens AR, Hollar D, Casey DE (2005) Schizophre-nia and increased risks of cardiovascular disease. American heart journal150(6):1115–1121[23] Weiner M, Warren L, Fiedorowicz JG (2011) Cardiovascular morbidity andmortality in bipolar disorder. Annals of clinical psychiatry: official journalof the American Academy of Clinical Psychiatrists 23(1):40[24] Pratt LA, Ford DE, Crum RM, Armenian HK, Gallo JJ, Eaton WW (1996)Depression, psychotropic medication, and risk of myocardial infarction:prospective data from the baltimore eca follow-up. Circulation 94(12):3123–3129[25] Kemp AH, Quintana DS, Gray MA, Felmingham KL, Brown K, Gatt JM(2010) Impact of depression and antidepressant treatment on heart ratevariability: a review and meta-analysis. Biological psychiatry 67(11):1067–1074[26] Tan G, Dao TK, Farmer L, Sutherland RJ, Gevirtz R (2011) Heart ratevariability (hrv) and posttraumatic stress disorder (ptsd): a pilot study.Applied psychophysiology and biofeedback 36(1):27–35[27] Chalmers JA, Quintana DS, Abbott MJ, Kemp AH, et al (2014) Anxietydisorders are associated with reduced heart rate variability: a meta-analysis.Frontiers in psychiatry 5:80[28] Quintana DS, Westlye LT, Kaufmann T, Rustan Ø, Brandt CL, HaatveitB, Steen NE, Andreassen OA (2016) Reduced heart rate variability inschizophrenia and bipolar disorder compared to healthy controls. Acta Psy-chiatrica Scandinavica 133(1):44–52 [29] Bai Y, Hibbing P, Mantis C, Welk GJ (2018) Comparative evaluation ofheart rate-based monitors: Apple watch vs fitbit charge hr. Journal of sportssciences 36(15):1734–1741[30] Levy DL, Sereno AB, Gooding DC, ODriscoll GA (2010) Eye tracking dys-function in schizophrenia: characterization and pathophysiology. In: Behav-ioral Neurobiology of Schizophrenia and Its Treatment, Springer, pp 311–347[31] Winograd-Gurvich C, Georgiou-Karistianis N, Fitzgerald PB, Millist L,White OB (2006) Ocular motor differences between melancholic and non-melancholic depression. Journal of affective disorders 93(1-3):193–203[32] Alghowinem S, Goecke R, Wagner M, Parker G, Breakspear M (2013) Eyemovement analysis for depression detection. In: 2013 IEEE InternationalConference on Image Processing, IEEE, pp 4220–4224[33] Dhuliawala M, Lee J, Shimizu J, Bulling A, Kunze K, Starner T, Woo W(2016) Smooth eye movement interaction using eog glasses. In: Proceed-ings of the 18th ACM International Conference on Multimodal Interaction,ACM, pp 307–311[34] Cano LAM, Beltr´an J, Navarro R, Garc´ıa-V´azquez MS, Castro LA (2017)Towards early dementia detection by oculomotor performance analysis onleisure web content. In: Proceedings of the 2017 ACM International JointConference on Pervasive and Ubiquitous Computing and Proceedings ofthe 2017 ACM International Symposium on Wearable Computers, ACM,pp 800–804[35] Critchley HD (2002) Electrodermal responses: what happens in the brain.The Neuroscientist 8(2):132–142[36] Schell AM, Dawson ME, Rissling A, Ventura J, Subotnik KL, Gitlin MJ,Nuechterlein KH (2005) Electrodermal predictors of functional outcome andnegative symptoms in schizophrenia. Psychophysiology 42(4):483–492[37] Lanata A, Greco A, Valenza G, Scilingo EP (2014) A pattern recognitionapproach based on electrodermal response for pathological mood identi-fication in bipolar disorders. In: 2014 IEEE International Conference onAcoustics, Speech and Signal Processing (ICASSP), IEEE, pp 3601–3605[38] Greco A, Valenza G, Lanata A, Rota G, Scilingo EP (2014) Electroder-mal activity in bipolar patients during affective elicitation. IEEE journal ofbiomedical and health informatics 18(6):1865–1873[39] Jandl M, Steyer J, Kaschka WP (2010) Suicide risk markers in major depres-sive disorder: a study of electrodermal activity and event-related potentials.Journal of Affective Disorders 123(1-3):138–149[40] Thorell LH, Wolfersdorf M, Straub R, Steyer J, Hodgkinson S, Kaschka W,Jandl M (2013) Electrodermal hyporeactivity as a trait marker for suicidalpropensity in uni-and bipolar depression. Journal of psychiatric research47(12):1925–1931[41] Roshanaei-Moghaddam B, Katon WJ, Russo J (2009) The longitudi-nal effects of depression on physical activity. General hospital psychiatry31(4):306–315 ental Health and Sensing 13 [42] Canzian L, Musolesi M (2015) Trajectories of depression: unobtrusive moni-toring of depressive states by means of smartphone mobility traces analysis.In: Proceedings of the 2015 ACM international joint conference on pervasiveand ubiquitous computing, ACM, pp 1293–1304[43] Saeb S, Zhang M, Karr CJ, Schueller SM, Corden ME, Kording KP, MohrDC (2015) Mobile phone sensor correlates of depressive symptom severityin daily-life behavior: an exploratory study. Journal of medical Internetresearch 17(7):e175[44] Wang R, Aung MS, Abdullah S, Brian R, Campbell AT, Choudhury T,Hauser M, Kane J, Merrill M, Scherer EA, et al (2016) Crosscheck: to-ward passive sensing and detection of mental health changes in people withschizophrenia. In: Proceedings of the 2016 ACM International Joint Con-ference on Pervasive and Ubiquitous Computing, ACM, pp 886–897[45] Nilsonne ˚A (1987) Acoustic analysis of speech variables during depressionand after improvement. Acta Psychiatrica Scandinavica 76(3):235–245[46] Moore E, Clements M, Peifer J, Weisser L (2004) Comparing objectivefeature statistics of speech for classifying clinical depression. In: The 26thAnnual International Conference of the IEEE Engineering in Medicine andBiology Society, IEEE, vol 1, pp 17–20[47] Ozdas A, Shiavi RG, Silverman SE, Silverman MK, Wilkes DM (2004)Investigation of vocal jitter and glottal flow spectrum as possible cues fordepression and near-term suicidal risk. IEEE Transactions on BiomedicalEngineering 51(9):1530–1540[48] Faurholt-Jepsen M, Munkholm K, Frost M, Bardram JE, Kessing LV (2016)Electronic self-monitoring of mood using it platforms in adult patients withbipolar disorder: a systematic review of the validity and evidence. BMCpsychiatry 16(1):7[49] Lu H, Frauendorfer D, Rabbi M, Mast MS, Chittaranjan GT, CampbellAT, Gatica-Perez D, Choudhury T (2012) Stresssense: Detecting stress inunconstrained acoustic environments using smartphones. In: Proceedings ofthe 2012 ACM Conference on Ubiquitous Computing, ACM, pp 351–360[50] Muaremi A, Gravenhorst F, Gr¨unerbl A, Arnrich B, Tr¨oster G (2014) As-sessing bipolar episodes using speech cues derived from phone calls. In:International Symposium on Pervasive Computing Paradigms for MentalHealth, Springer, pp 103–114[51] Abdullah S, Matthews M, Murnane EL, Gay G, Choudhury T (2014) To-wards circadian computing: early to bed and early to rise makes some ofus unhealthy and sleep deprived. In: Proceedings of the 2014 ACM inter-national joint conference on pervasive and ubiquitous computing, ACM, pp673–684[52] Frost M, Doryab A, Faurholt-Jepsen M, Kessing LV, Bardram JE (2013)Supporting disease insight through data analysis: refinements of themonarca self-assessment system. In: Proceedings of the 2013 ACM inter-national joint conference on Pervasive and ubiquitous computing, ACM, pp133–142 [53] Matthews M, Murnane E, Snyder J, Guha S, Chang P, Doherty G, GayG (2017) The double-edged sword: A mixed methods study of the inter-play between bipolar disorder and technology use. Computers in HumanBehavior 75:288–300[54] Alvarez-Lozano J, Osmani V, Mayora O, Frost M, Bardram J, Faurholt-Jepsen M, Kessing LV (2014) Tell me your apps and i will tell you yourmood: correlation of apps usage with bipolar disorder state. In: Proceedingsof the 7th International Conference on PErvasive Technologies Related toAssistive Environments, ACM, p 19[55] Wolff EA, Putnam FW, Post RM (1985) Motor activity and affective ill-ness: the relationship of amplitude and temporal distribution to changes inaffective state. Archives of General Psychiatry 42(3):288–294[56] Walther S, Stegmayer K, Horn H, Razavi N, M¨uller TJ, Strik W (2015)Physical activity in schizophrenia is higher in the first episode than in sub-sequent ones. Frontiers in psychiatry 5:191[57] John D, Freedson P (2012) Actigraph and actical physical activity monitors:a peek under the hood. Medicine and science in sports and exercise 44(1Suppl 1):S86[58] Wu W, Dasgupta S, Ramirez EE, Peterson C, Norman GJ (2012) Classi-fication accuracies of physical activities using smartphone motion sensors.Journal of medical Internet research 14(5):e130[59] Osmani V, Maxhuni A, Gr¨unerbl A, Lukowicz P, Haring C, Mayora O(2013) Monitoring activity of patients with bipolar disorder using smartphones. In: Proceedings of International Conference on Advances in MobileComputing & Multimedia, ACM, p 85[60] Beiwinkel T, Kindermann S, Maier A, Kerl C, Moock J, Barbian G, R¨osslerW (2016) Using smartphones to monitor bipolar disorder symptoms: a pilotstudy. JMIR mental health 3(1):e2[61] Aharony N, Pan W, Ip C, Khayal I, Pentland A (2011) Social fmri: Inves-tigating and shaping social mechanisms in the real world. Pervasive andMobile Computing 7(6):643–659[62] Ben-Zeev D, Wang R, Abdullah S, Brian R, Scherer EA, Mistler LA, HauserM, Kane JM, Campbell A, Choudhury T (2015) Mobile behavioral sens-ing for outpatients and inpatients with schizophrenia. Psychiatric services67(5):558–561[63] Faurholt-Jepsen M, Vinberg M, Frost M, Debel S, Margrethe ChristensenE, Bardram JE, Kessing LV (2016) Behavioral activities collected throughsmartphones and the association with illness activity in bipolar disorder.International journal of methods in psychiatric research 25(4):309–323[64] Reece AG, Reagan AJ, Lix KL, Dodds PS, Danforth CM, Langer EJ (2017)Forecasting the onset and course of mental illness with twitter data. Scien-tific reports 7(1):13,006[65] Abdullah S, Murnane EL, Costa JM, Choudhury T (2015) Collective smile:Measuring societal happiness from geolocated images. In: Proceedings ofthe 18th ACM Conference on Computer Supported Cooperative Work &Social Computing, ACM, pp 361–374 ental Health and Sensing 15 [66] Reece AG, Danforth CM (2017) Instagram photos reveal predictive markersof depression. EPJ Data Science 6(1):15[67] Breiman L (2000) Randomizing outputs to increase prediction accuracy.Machine Learning 40(3):229–242[68] Lane ND, Mohammod M, Lin M, Yang X, Lu H, Ali S, Doryab A, BerkeE, Choudhury T, Campbell A (2011) Bewell: A smartphone application tomonitor, model and promote wellbeing. In: 5th international ICST confer-ence on pervasive computing technologies for healthcare, pp 23–26[69] Abdullah S, Matthews M, Frank E, Doherty G, Gay G, Choudhury T (2016)Automatic detection of social rhythms in bipolar disorder. Journal of theAmerican Medical Informatics Association 23(3):538–543[70] Mart´ınez HP, Yannakakis GN (2014) Deep multimodal fusion: Combiningdiscrete events and continuous signals. In: Proceedings of the 16th Interna-tional conference on multimodal interaction, ACM, pp 34–41[71] Consolvo S, McDonald DW, Toscos T, Chen MY, Froehlich J, Harrison B,Klasnja P, LaMarca A, LeGrand L, Libby R, et al (2008) Activity sensingin the wild: a field trial of ubifit garden. In: Proceedings of the SIGCHIconference on human factors in computing systems, ACM, pp 1797–1806[72] Costa J, Adams AT, Jung MF, Guimbreti`ere F, Choudhury T (2016) Emo-tioncheck: leveraging bodily signals and false feedback to regulate our emo-tions. In: Proceedings of the 2016 ACM International Joint Conference onPervasive and Ubiquitous Computing, ACM, pp 758–769[73] Aung MH, Matthews M, Choudhury T (2017) Sensing behavioral symptomsof mental health and delivering personalized interventions using mobile tech-nologies. Depression and anxiety 34(7):603–609[74] Asselbergs J, Ruwaard J, Ejdys M, Schrader N, Sijbrandij M, Riper H(2016) Mobile phone-based unobtrusive ecological momentary assessment ofday-to-day mood: an explorative study. Journal of medical Internet research18(3):e72[75] Ahmed SI, Haque MR, Chen J, Dell N (2017) Digital privacy challengeswith shared mobile phone use in bangladesh. Proceedings of the ACM onHuman-Computer Interaction 1(CSCW):1–20[76] Shilton K, Sayles S (2016) ” we aren’t all going to be on the same page aboutethics”: Ethical practices and challenges in research on digital and socialmedia. In: 2016 49th Hawaii International Conference on System Sciences(HICSS), IEEE, pp 1909–1918[77] De Montjoye YA, Hidalgo CA, Verleysen M, Blondel VD (2013) Unique inthe crowd: The privacy bounds of human mobility. Scientific reports 3:1376[78] Sweeney L (2000) Simple demographics often identify people uniquely.Health (San Francisco) 671:1–34[79] Brush A, Krumm J, Scott J (2010) Exploring end user preferences for lo-cation obfuscation, location-based services, and the value of location. In:Proceedings of the 12th ACM international conference on Ubiquitous com-puting, ACM, pp 95–104[80] Tushar AK, Antara IJ, Das D, Chandra P, Soron TR, Haque MM, AhamedSI, Ahmed SI (2020) “we need more power to stand up”: Designing to6 Mental Health and Sensing