Mobile-Based Experience Sampling for Behaviour Research
Veljko Pejovic, Neal Lathia, Cecilia Mascolo, Mirco Musolesi
MMobile-Based Experience Sampling forBehaviour Research
Veljko Pejovic, Neal Lathia, Cecilia Mascolo and Mirco Musolesi
Abstract
The Experience Sampling Method (ESM) introduces in-situ sampling ofhuman behaviour, and provides researchers and behavioural therapists with ecolog-ically valid and timely assessments of a person’s psychological state. This, in turn,opens up new opportunities for understanding behaviour at a scale and granularitythat was not possible just a few years ago. The practical applications are many, suchas the delivery of personalised and agile behaviour interventions. Mobile computingdevices represent a revolutionary platform for improving ESM. They are an insepa-rable part of our daily lives, context-aware, and can interact with people at suitablemoments. Furthermore, these devices are equipped with sensors, and can thus takepart of the reporting burden off the participant, and collect data automatically. Thegoal of this survey is to discuss recent advancements in using mobile technologiesfor ESM (mESM), and present our vision of the future of mobile experience sam-pling.
Veljko PejovicFaculty of Computer and Information Science, University of Ljubljana, Sloveniae-mail: [email protected] LathiaComputer Laboratory, University of Cambridge, United Kingdome-mail: [email protected] MascoloComputer Laboratory, University of Cambridge, United Kingdome-mail: [email protected] MusolesiDepartment of Geography, University College London, United Kingdome-mail: [email protected] 1 a r X i v : . [ c s . H C ] A ug Veljko Pejovic, Neal Lathia, Cecilia Mascolo and Mirco Musolesi
Human behaviour often depends on the context in which a person is. This con-text is described by our physical environment, for example, a semantic location,such as home or work, our physical state, such as running or sleeping, but alsoby our internal state, for example, our cognitive load. The manifestations of hu-man behaviour are complex, and can be observed through our actions, thoughts,and emotions, to name a few descriptors. For psychologists, understanding humanbehaviour necessitates capturing behaviour as it happens. Initial methods of captur-ing behaviour included lab studies, where participants were placed in an artificialsituation and closely monitored, as well as retrospective interview studies, whereparticipants were asked to recall their past experiences. However, since behaviourdepends on the context, which is often much richer than anything that can be createdin the lab, these studies cannot be used to faithfully replicate natural behaviour.
TheExperience Sampling Method (ESM) was developed to capture human behaviour asit happens [20].The essence of ESM lies in occasional querying of users who then provide im-mediate answers to questions asked. The method avoids both direct interaction witha researcher/therapist, as well as artificial lab-made environments. As such, ESM-obtained data are, first of all, recorded in the context, and thus of higher ecolog-ical validity than data obtained by legacy means of collection. Second, they arerecorded closely after the moment of querying, minimising the retrospective biassymptomatic to data harvested by earlier methods. Furthermore, ESM allows long-term querying and longitudinal studying of participants, and may be able to capturesamples during infrequently occurring events.The original approach to sampling users in an ESM study included a pro-grammable beeper that indicates times at which a sample should be taken, and apaper diary that participants fill out once the beeper rings [20]. Different forms ofdata collection and querying, such as calling users on their mobile phones, or usinga personal digital assistant (PDA) device to collect data, have been used in earlierstudies [34]. A common characteristic of these studies is the need to deploy un-conventional technology, such as beepers and diaries, and train the participants touse them. Furthermore, itself context-oblivious, the technology did not allow therecognition of and sampling at “interesting” moments only. Finally, relying on thehonesty of users’ self-reporting is one of the major drawbacks of the traditionalESM approach [8].In the past decade, mobile computing devices, including smartphones and wear-ables such as smart watches, have become a part of everyday life. Integration withthe user and high computing and sensing capabilities render these devices a rev-olutionary platform for social science research [37], where mobile computing canbe used to gather fine-grain personal data from a large number of individuals. Theavailability of these personal data has contributed to the emergence of the new re-search field of computational social science [31].In this survey we present an overview of user behaviour sampling via mobilecomputing devices, and pay particularly attention to practical issues associated with obile-Based Experience Sampling for Behaviour Research 3
Survey data
Data storage serverApplication executableContext sensing Context-triggered survey
Sensor data
Fig. 1
Experience sampling on a smartphone. An ESM application executable is downloaded onthe phone. The application manages sensing, and processes the sensed data in order to infer inter-esting moments when user’s data should be captured. When such moments are recognised, a useris prompted to fill in a survey. Data collected from the user, along with the data sensed by mobilesensors, are uploaded to a data storage server for further analysis. designing and deploying behavioural studies using mobile ESM (mESM). First, weidentify the main novelties that mESM brings to the table, of which remote sensingis the one poised to induce a major change to the current practice. Then, we surveythe most popular open-source tools that streamline study design and deployment, bylifting the burden of mobile device programming from researchers and therapists,who might have a limited set of technological skills. We then discuss the challengesin running an mESM study, including recruiting participants, ensuring non-biasedexperience sampling, retaining participants, and handling technological limitationsof mobile devices. Finally, we present our vision of mESM, which includes adap-tive sampling according to a user’s lifestyle, delivery of tailored behaviour changeinterventions based on the sampled data, and proactive reasoning and interaction inthe manner of anticipatory computing.
Mobile devices are poised to completely transform numerous aspects of experiencesampling in behavioural psychology. Study design, participant recruitment, data col-lection, and the sheer amount of data gathered by mESM are incomparable to thesame aspects encountered with the legacy means of experience sampling. As an il-lustrative example, in Figure 1 we show a smartphone-based mESM application.The application is distributed as an executable file, possibly via an application store,to a large number of participants owning commodity smartphones. A personalisedinstance of the application is then run at each of the phones, where it harnessesphone’s sensing ability to recognise the situation in which a user is, and should the
Veljko Pejovic, Neal Lathia, Cecilia Mascolo and Mirco Musolesi situation be of interest, signals a user to fill in a survey. The user-provided informa-tion is then, together with the data sensed by smartphone’s sensors, dispatched to acentralised server where it can be analysed.Smartphones-based mESM studies improve the traditional beeper and paper formstudies in a few important ways. First, unlike beepers and diaries, smartphones arealready a part of users’ lives, and do not interfere with users’ lifestyle. With mESMstudies we are “piggybacking” in a sense on an already used device, reducing theburden on the participant to carry an additional device, and lowering the cost of thestudy. Moreover, using a conventional device, participants are less likely to be em-barrassed about completing a questionnaire [50]. In addition, each ESM study canbe carried out as separate mobile application, distributed over a large number of de-vices via application stores such as Google Play and Apple App Store. This enablesunprecedented scalability and parallelisation of experience sampling studies.Second, modern mobile devices are equipped with a range of sensors, from GPSto light, proximity, and movement sensors. Therefore, unlike beepers, smartphones“know” the context in which a user is. As shown in the example (Figure 1), this canaugment data collection. ESM studies often aim to capture user experience withina certain situation, for example, whenever a user leaves home. Beepers use pre-programmed times, and in case events cannot be reliably forecast, a user’s depar-ture time from a certain location is a likely candidate for such an event, we haveno means of ensuring that relevant events are captured. Smartphones, on the otherhand, can infer the context from sensor readings, and then prompt the user to fill inthe survey as the desired context is happening. Location-dependent reminders, forexample, are already a part of commercial applications [16]. In addition, the maincaveat of the beeper-based ESM is its reliance on the honesty of self reports. De-vice location, user’s activity, and their social circle, can be inferred with the helpof smartphone sensors. Numerous aspects of context can be reported directly by thesmartphone, avoiding user-induced errors in the data. Finally, the sensed context canbe directly relevant to an ambulatory assessment of a user’s psycho-physical state.Previous studies put a great effort to combine participants’ diary entries with theirheart rate or blood pressure, for example [10]. Nowadays, devices such as smart-watches, which come integrated with with galvanic skin response and heart ratesensors, enable holistic ambulatory assessment/ESM studies at scale.
Mobile Sensing for ESM:
Modern smartphones, almost without exception,feature location, orientation, acceleration, and light sensors, together withcameras and a microphone. High-end models host over a dozen of differentsensors, including barometer, heart rate and gesture sensors. Combined withhigh computing power provided by today’s phones’ multicore CPUs, smart-phones represent an attractive platform for real-time context inference. Formost of the day phones are carried by their owners, thus sensor data closelyreflects actual users’ behaviour and the change of the context around the user.With the help of machine learning, personalised models of different contex-tual aspects can be built on top of the collected data. Phones are routinely obile-Based Experience Sampling for Behaviour Research 5 exploited to infer users’ semantic location (home, work) via GPS-assisted mo-bility models. Sensor data from a phone’s built-in accelerometer can be usedto infer a person’s physical activity. Sounds captured by the built-in micro-phone can be processed to infer if a conversation is taking place in a user’svicinity, but also to infer a user’s stress level and emotional states [38, 44, 32].A Bluetooth chip, itself merely a short-range communication enabler, can beused to infer social encounters of a phone owner [43]. A number of high-leveldescriptors of human behaviour can be inferred by combining the sensor datacoming from different sensors, including contextual information from onlinesocial networks [26, 35]. However, we should not forget that above all, to-day’s phones are communication devices providing always-on voice and dataconnectivity. Thus, for the first time, with mobile computing ESM researchershave a possibility to design truly context-aware studies, to get real-time infor-mation about the context in which the participants are, and to adjust samplingstrategies on the fly.
The design, implementation and deployment of experience sampling studies via mo-bile devices requires expertise that is not confined to the traditional social sciencetraining. A smartphone-based mESM study, for example, entails a significant pro-gramming effort in building the application and managing mobile sensing, as wellas the construction of sophisticated machine learning models for context inference,and ensuring reliable data transfer from remote devices to a centralised server. Notonly are these tasks often outside the psychological researchers’ and therapists’ ex-pertise, they also result in a lot of replicated effort for each new study.Table 1 lists some of the frameworks developed by the research community inorder to streamline the process of conducting mobile experience sampling stud-ies . The first frameworks preceded the smartphone era. The Experience SamplingProgram (ESP) runs on Palm Pilot PDA devices, and lacks the sophisticated con-text awareness introduced in later frameworks [4]. However, the ESP was the firstframework to introduce an authoring tool for designing experience sampling ques-tionnaires for mobile devices. The tool also lets a study designer define a logic fortiming the questionnaire prompts. Compared to the traditional beeper and diary stud- Every effort has been made to provide truthful descriptions of the listed mESM frameworks,however, due to limited documentation and publications related to some of the frameworks thelisted properties should be taken with caution. The goal of this article is to suggest guidelines for future research in the field, thus we concentrateon free open-source software developed in academia, as such software can serve as a basis for nextgeneration mESM frameworks. Commercial products for supporting mESM are outside of thescope of our article. Veljko Pejovic, Neal Lathia, Cecilia Mascolo and Mirco Musolesi T a b l e F r a m e w o r k s f o r bu il d i ng m ob il ee xp e r i e n ce s a m p li ng s t ud i e s . N a m e UR L P l a tf o r m Su r v e y ss upp o r t C o n t e x t s e n s i n g M o b il ec o m p o n e n t S er v erc o m p o n e n t D a t aa n a l y s i s D e s cr i p t i o n E SP [ ] . e xp e r i e n ce - s a m p li ng . o r g P a l m P il o t Y e s N o Y e s Y e s N o T h e fi r s t m E S M a pp , P C - b a s e d s t udyd e s i gn t oo l . M y E xp e r i e n ce [ ] m y e xp e r i e n ce . s ou r ce f o r g e . n e t P o c k e t P C Y e s Y e s Y e s N o N o I n t r odu ce s c on t e x t - a w a r e s a m p li ng . P s y c h L og [ ] s ou r ce f o r g e . n e t/ p r o j ec t s / p s y c h l og W i ndo w s m ob il e Y e s Y e s Y e s N o N o S uppo r t s e x t e r n a l s e n s o r s ( e . g . E C G ) . A nd W e ll n e ss [ ] N o t a v a il a b l e ( J u l y2015 ) A nd r o i d Y e s Y e s Y e s Y e s Y e s H ea lt h ca r e - o r i e n t e d , d a t a v i s u a li s a ti on . E m o ti on S e n s e [ , ] e m o ti on s e n s e . o r g A nd r o i d Y e s ( un r e l ea s e d ) Y e s Y e s Y e s ( un r e l ea s e d ) N o E m o ti on s e n s i ng a pp , w it hop e n - s ou r ce li b r a r i e s . O h m a g e [ ] oh m a g e . o r g A nd r o i d /i O S Y e s Y e s Y e s Y e s Y e s A ll o w s h i gh - l e v e l c on t e x ti n f e r e n ce . f un f[ ] . f un f . o r g A nd r o i d N o Y e s Y e s N o N o A r i c h s e n s i ng fr a m e w o r k . O p e n D a t a K it [ ] . op e nd a t a k it . o r g A nd r o i d Y e s Y e s Y e s Y e s Y e s D a t ac o ll ec ti on t oo lt a r g e ti ngnon - e xp e r t u s e r s . P ac og it hub . c o m / goog l e / p ac o A nd r o i d /i O S Y e s Y e s Y e s Y e s N o A n e x t e n s i b l e fr a m e w o r k f o r qu a n ti fi e d s e l f e xp e r i m e n t s . P u r p l e R obo tt ec h . c b it s . no r t h w e s t e r n . e du / pu r p l e -r obo t A nd r o i d N o Y e s Y e s Y e s N o A fr a m e w o r k f o r s e n s i ng a nd s e n s o r- b a s e d ac ti on i ng . S e n S o c i a l [ ] c s . bh a m . ac . uk / a x m / s e n s o c i a l A nd r o i d N o Y e s Y e s Y e s N o A li b r a r y f o r j o i n t s a m p li ngo f O S N a nd s e n s o r d a t a s t r ea m s . obile-Based Experience Sampling for Behaviour Research 7 ies, ESP-based studies combine signalling and data collection on the same device,yet, PDA devices have never achieved mass popularity needed for large-scale ESMstudies in the wild.Recognising context was the most important missing feature in traditional ESMstudies. Event-contingent sampling, where the time of sampling depends on the con-text or an event in which a user is, is of particular interest for psychological stud-ies [46]. Such sampling is important in case target events are rare, short-lasting, orunpredictable, in which case periodic sampling might completely miss them. Forexample, Cote and Moskowitz investigated the impact of the “big five” personal-ity traits on the relationship between interpersonal behaviour and affect [7]. Theparticipants were instructed to fill out a questionnaire following each interpersonalinteraction. Without context-aware devices, Cote and Moskowitz used beeper to pe-riodically remind participants to keep up with the study, but telling them that theanswers should be provided only after an interaction has happened. However, thecorrectness of this approach, particularly the timeliness of harvested data, dependson the users’ compliance with the rules of the study. The study designers have nomeans of checking whether, and when, interpersonal interactions have happened.The MyExperience framework [13], built upon an earlier context-aware mESMtool developed by Intille et al. [23], runs on Pocket PC and lets researchers designstudies that embrace context awareness provided by mobile sensing. On one side,sensor data can be passively logged on the user device and uploaded to a server, onthe other, the data can be processed on the phone to infer the context and triggerevent-contingent sampling if needed. As one of the first examples of an mESMframework, MyExperience’s context triggering relies on raw sensor data, i.e., it doesnot perform any inference in order to extract higher level information. For example,the framework supports sampling when a user moves from one mobile cellular basestation to another, but cannot recognise if a user arrived at a semantically significantlocation, say his/her workplace.MyExperience and ESP set a foundation for modern experience sampling frame-works, while a wider adoption of mESM frameworks came with the rise of thesmartphone that enabled remote data gathering without requiring user actions, andcontext-aware user querying. The first Apple iPhone, released in 2007 and packedwith numerous high-resolution sensors, marked a revolution in mobile sensing. De-vices from different vendors followed, often running Android OS that enabled finercontrol over sensing than ever before. Modern smartphone sensing, however, hasto balance between limited energy resources available on the phone, and the needfor fine-grained data from multiple sensors. In addition, smartphone’s sensors werenot originally conceived for continuous sampling. Sensing and data collection man-agement become a new pressing issue for mobile computing. Sensing frameworkssuch as
ESSensorManager (a part of the Emotion Sense project) aim to abstract thedetails about data acquisition and collection from an application developer and au-tomate sensing as per predefined policies [30]. The funf framework adds an optionof basic survey data collection, and was used in a detailed 15-month long study of130 participants’ social and physical behaviour [3]. The study provided an in-depthinvestigation of the connection between individuals’ social behaviour and their fi-
Veljko Pejovic, Neal Lathia, Cecilia Mascolo and Mirco Musolesi nancial status, and the effects of one’s network in decision making. The power ofsensed data was further demonstrated when on top of funf the authors built an inter-vention that not only sampled users’ behaviour, but also influenced users to exercisemore.Raw data from mobile sensors can be difficult to interpret in terms that are of di-rect interest when sampling human behaviour. High-level inferences often need to bemade before sampled experience becomes valuable for researchers and therapists.
Purple Robot ’s authors claim that their framework supports statistical summariesof the user’s communication patterns, including phone logs and text-message tran-scripts.
Ohmage [45], a platform for participatory sensing and ESM studies, hostsa few data classifiers that can infer concepts such as mobility and speech. A psy-chology practitioner faces a large barrier between raw data from sensors that arerelated to users’ behaviour, e.g., their movement, and the high-level labels of thosebehaviours, e.g., if people are walking or driving a car. It is crucial for mESMframeworks to abstract the sensing, in this case accelerometer and GPS sensorsampling, data processing, in this case extracting acceleration variance and GPS-reported speed, and machine learning, in this case classify a mobility mode, anddeliver high-level information. Recently, Google released its activity recognitionAPI for Android, enabling easy, albeit crude, inference of user’s activity state [1].Besides built-in smartphone sensors,
PsychLog [15] and
Open Data Kit (ODK) [19]frameworks provide support for external sensors that can be attached to a mobile de-vices, greatly enhancing the utility of ESM for ambulatory assessment. Moreover,for understanding human behaviour, any source of human-related information canbe a sensor. In particular, social interactions are for a large part conducted overonline social networks (OSNs), and monitoring OSNs is increasingly becoming afocus of social science studies. For instance, this information-rich sensor can bemerged with the physical context sensors, in order to uncover socio-environmentalrelationships. An example of system supporting this type of real-time data fusion is
SenSocial , a distributed (residing on mobiles and a centralised server) middlewarefor merging OSN-generated and physical sensor data streams [35].A successful mESM framework abstracts mobile application programming froman intervention developer, yet exposes enough functionality so that a variety of stud-ies are supported. Early on, MyExperience used an XML-based interface throughwhich study developers can define sensor data to be collected, survey questionsand triggers that will alert users to fill in the surveys. Although close to a natu-ral language, XML scripts are not an ideal means of describing a potentially com-plex mESM application. Targeting primarily less tech savvy users in the developingworld, ODK puts an emphasis on hassle-free study design process [19]. The frame-work introduces a survey and sensor data collection authoring tool that enables drag-and-drop study design. ODK is further tailored for non-expert designers and studyparticipants by supporting automated data upload, storage and cloud transfer, aswell as automated phone prompts that users respond to with keypad presses. TheProject Authoring tool that comes with the Ohmage framework guides a study de-signer through a project definition process, and outputs an XML definition of thestudy. Ohmage also features tools such as Explore Data, Interactive and Passive obile-Based Experience Sampling for Behaviour Research 9
Dashboards, and Lifestreams, that enable in-depth analysis and visualisation of col-lected data. In particular, Lifestreams use statistical inference on raw collected datain order to examine behavioural trends and answer questions such as “how muchtime a user spends at work/home?”.
Contextual data collection, rapid prototype design, study scalability, and automatedresult analysis are just some of the ways in which mobile devices revolutionise thetraditional ESM. However, certain original limitations of the method are still presenteven with this new technology. How to capture experience without interfering withthe participant’s lifestyle, and how to sample relevant moments when the user’slifestyle is highly varying and unpredictable are some of the questions existing sincethe ESM was introduced. Some other issues, such as user recruitment become moreprevalent now that a study can be distributed in form of an application that could berun on millions of devices. Furthermore, the new platform introduced novel techni-cal challenges that impact the way studies should be designed.
The Internet empowered social scientists with an easy access to a large and diversepool of participants, alleviating the predominant issue of running psychologicalstudies on a small group of college students [17]. Yet, recruitment in initial mESMstudies saw little benefit from the Internet, since the participation was throttled, justlike in the case of the older beeper technology, by the availability of the support-ing hardware, i.e., Pocket PCs. Nowadays, with 1.5 billion users the smartphone isone of the most ubiquitous devices on the planet. Consequently, smartphone-basedmESM studies can be distributed at an unprecedented scale.All major smartphone operating systems, such as Android, iOS and WindowsMobile, have their corresponding online application stores. With these stores as dis-tribution channels, the pool of study participants is no more confined to a certainpopulation that a study designer can reach. Despite concerns about the diversity ofparticipants recruited through the Internet, Gosling et al. show that such a sampleis more representative of the actual demographics than a sample recruited throughtraditional means [17]. Note, however, that the Internet has been around longer thansmartphones, and has penetrated almost all segments of the society. Still, a rapid risein smartphone ownership promises to erase any demographic biases that currentlymay exist in smartphone usage.To a potential participant, a smartphone-based mESM provides an additionalbenefit of anonymity, as a user does not have to disclose her real name, nor needsto meet the people/organisation running the study. On the down side, researchers have to sacrifice the close control over who the study participants are. For exam-ple, there is no reliable way to confirm that a person’s age is truthfully reported,potentially allowing minors to run adult-only studies. Mobile sensing can somewhatameliorate the problem of false reporting, as it provides information about users’activity, movement, geographic location, communication patterns and others. It hasbeen shown that such information reflects users’ age, gender, social status [12]. Be-sides assisting in the pruning of false reports, the link between sensor data and thedemographics can be used to selectively target a certain demographic group, or totailor the study according to different groups, e.g., adjusting sampling times accord-ing to local customs, sending different questions to people belonging to differentsocial groups.A low entry barrier that smartphone mESM applications provide, also means thatleaving a study is easy – a user just has to uninstall or ignore the application. Usu-ally only a percentage of users is active after the first initial period. Furthermore,on global application markets mESM study applications compete with hundredsof thousands of useful and fun applications. One way of attracting and retaining awide audience for an mESM study is by providing some kind of information backto the user. In Emotion Sense, an mESM application that captures emotional stateand contextual sensor information, participant retention is achieved through gam-ification and provision of self-reflecting information about the user [29]. EmotionSense invites a user to “unlock” different parts the application by providing furtherexperience samples.Another means of attracting and retaining users is through remuneration. Ama-zon Mechanical Turk is a popular crowdsourcing marketplace where requesters postjobs to be completed by workers . The jobs typically consists of simple, well-definedtasks for which computers are not suitable, such as data verification, image analysisand data collection. Workers are paid a previously agreed sum of money per com-pleted task. Any adult person, from any continent, can become a worker. In [33],Manson and Suri discuss the opportunities for conducting behavioural research onAmazon’s Mechanical Turk. A wide pool of participants for a study and a paymentsystem that enforces job completion are emphasised as the main advantages of theMechanical Turk. On the other hand, artificial automatic workers – bots – can beused by workers to fake study results without actually running the study. In addi-tion, the Mechanical Turk workers are not representative of general, even online,demographics. We believe, however, that mobile sensing can be used to ameliorateboth problems. Artificial behaviour can be detected through unusual activity andmovement patterns, while as explained earlier, sensed data can be used to infer theparticipants demographics.
Smartphone-based mESM applications run on devices that are an inseparable partof participants’ lives. Thus, it is crucial for a sampling schedule to be in harmony obile-Based Experience Sampling for Behaviour Research 11 with the user’s lifestyle. A well designed interruption schedule can help in bothretaining users, but also in fulfilling the true role of experience sampling – recordingmomentary experience – as participants not wishing to be interrupted are likely tointroduce the recall bias by delaying their answers until they find a suitable momentto respond [36].Interactivity is in the core of human behaviour, as we balance between work-ing on a task and switching to other pressing issues. The mobile phone makes ourlives increasingly interactive as notifications delivered via mobile devices becamea dominant means of signalling possible tasks switching events. In mESM studiesmobile notifications are used to prompt users to fill in sampling surveys. The timingof notifications is important, since in case a notification arrives in an opportune mo-ment for interruption the user reacts to it quickly, and fills in a survey with timelydata. Several research studies investigated mobile notification scheduling in order toidentify these opportune moments, and found that the context in which a person is,to a large extent, determines if a user is interruptible or not [22, 49]. Equipped withsensors, a mobile device can infer this context. Ho and Intille, for example, showthat external on-body accelerometers can detect moments of activity transitions, andthat in such moments users react to an interruption more favourably [22].In [41] a 20-person two-week study of mobile interruptibility shows how datafrom built-in smartphone sensors relates to user’s attentiveness to mobile interac-tions in form of notifications. The study demonstrates that a personalised model ofthe sensed data – interruptibility relationship can be built, after which the authorsextract the sensor modalities that describe user interruptibility, including accelera-tion, location and time information, and implement personalised machine learningmodels that, depending on the given sensor input, infer interruptibility. The find-ings are funnelled into a practical system termed
InterruptMe – an Android libraryfor notification management, that informs an overlying application about oppor-tune moments in which to interrupt a user . Machine learning-based models thatInterruptMe builds are refined over time. However, ESM studies are limited by thenumber of samples that are taken from a single person over a period of time, thusthe phone should learn about when to interrupt a user with as few training samplesas possible. Kapoor and Horvitz propose a decision-theoretic approach for minimis-ing the number of samples one needs to take from a user in order to build a reliablemodel of that persons interruptibility [24]. Finally, the InterruptMe study finds thatinterruption moments cannot be considered in isolation, and that users’ sentimenttowards an interruption depends on the recently experienced interruption load. Thismay become a limiting factor as the number of applications, and consequently noti-fications, that a user gets on her phone grows. InterruptMe is available as a free open-source software at bitbucket.org/veljkop/intelligenttrigger2 Veljko Pejovic, Neal Lathia, Cecilia Mascolo and Mirco Musolesi
A decision about when, or under which conditions, a notification to complete asurvey should be fired is not only important for improved user interaction and com-pliance, it also has a crucial effect on the data that will be harvested.Studies have, for example, been designed to collect data at random intervals [4] orwhen smartphone sensors acquire readings of a particular value[13]. While the latteris often motivated by directly tying a device state with a device-related assessment(e.g., plugging in the phone triggering questions about phone charging [13]), both ofthese methods have been used by researchers to make inferences and test hypothesesabout broad, non-device specific aspects of participants’ behaviours, such as dailyevents and moods [5] and sustainable transportation choices [14].This methodology assumes that the design choice of which trigger to use will notaffect (or, indeed, will even augment the accuracy of) the contextual data that canbe used to learn about participants. In doing so, these studies do not take into ac-count the effect that the designed sampling strategy has on the conclusions they inferabout participants’ behaviours. However, these behaviours are likely to be habitualor, more broadly, variantly distributed across each day. For example, since peoplemay split the majority of their time between home and work, sampling randomlyis likely to fail capturing participants in other locations. While this could easily besolved by using survey triggering that is conditioned on the value of a user’s loca-tion (from here on termed location-based triggering ), it is not clear how doing soaffects sampling from the broader set of sensors that researchers may be collectingdata from (i.e., how would location-based sampling bias the data about participantsactivity levels?).In [29], Lathia et al. study the effect of mESM design choices on the inferencesthat can be made from participants’ sensor data, and on the variance in survey re-sponses that can be collected from them. In particular the authors examined thequestion: are the behavioural inferences that a researcher makes with a time ortrigger-defined subsample of sensor data biased by the sampling strategy’s design?
The study demonstrates that different single-sensor sampling strategies will resultin what is refer to as contextual dissonance : a disagreement in how much differentbehaviours are represented in the aggregated sensor data.To analyse this, Lathia et al. examine the extent that studies’ design influences theresponse and sensor/behavioural data that researchers can collect from participantsin context-aware mESM studies. If the design of the experiments does not have anyinfluence on the data that are collected, we would expect that, on aggregate, the datagleaned from different designs would be consistent with one another. Instead, thestudy demonstrates that different single-sensor sampling strategies result in contex-tual dissonance, i.e., a disagreement in how much different behaviours are repre-sented in the aggregated sensor data. This conclusion is based on a 1-month, 22-participant mESM study that solicited survey responses about participants’ moodswhile collecting data from a set of sensors about their behaviour. This falls undertwo broad groups: obile-Based Experience Sampling for Behaviour Research 13 • Amount of Data.
Using different sensors to trigger notifications will directly im-pact the amount of data that researchers can collect. In this study, microphone-based triggers, which pop-up a survey only if a non-silent audio sample issensed, produce a higher per-user average number of notifications; conversely,the communication-based triggers, fired after a call/SMS received/sent event,produce the lowest average number of notifications per participant (5 . ± . • Response Data.
In addition, the study examines how the feelings of participantsvaried under different experimental conditions. In this case, the null hypothesis isthat the design of survey triggers does not bias the resulting sample of affect datathat is collected. This hypothesis is rejected, with varying levels of confidence,if the resulting p-values are small. In 4 of the 6 tests that were performed, it wasfound that the negative affect ratings (and 2 of 6 for the positive ratings) weresignificantly different from one another with at least 90% confidence. Uncoveringwhy this result has emerged calls for further research: it may be explained by thefact that some triggers were likely to be more obtrusive than others, thus affectingresponses.Finally, we point out that the above conclusion is based on a time-limited andsmall-scale study. Perhaps some of these challenges can be overcome by usinglarger populations for longer times. However, these results stand as a warning forresearchers to be mindful of in their future mobile experience sampling studies.
A smartphone is a multi-purpose device used for voice, video, and text communica-tion, Web surfing, calendar management or navigation, among other applications.This versatility puts pressure on smartphone’s resources, and limits its usabilityfor experience sampling. Furthermore, unlike a conventional mobile application,an mESM application needs to be always-on, sense the context and sample users’experiences as necessary.One of the key constraints of many mobile sensing applications is a limited ca-pacity of a mobile device’s battery. Power-hungry sensors, such as a GPS chip, werenot envisioned for frequent sampling. Adaptive sensing is a popular means of reduc-ing energy requirements of a mobile sensing application. Here, samples are takenless frequently, or with a coarser granularity, e.g., a user’s location is recorded with abase station ID, instead of with accurate GPS coordinates.
AndWellness framework,for example, lets study designers tune the balance between the sampling resolutionand power drain [21]. The same framework implements hierarchical sensor activa-tion, another means of minimising energy usage. This approach uses low-power,yet less accurate, sensors in order to infer if high-power, fine-grain sensors shouldbe turned on. An example of hierarchical sensing can be seen in AndWellness: achange in a device’s WiFi access point association serves as an indicator of user’s movement, and if movement is detected a GPS chip is activated. While Ohmage im-plements adaptive sensing to save energy during speech detection by adjusting thesampling rate depending on the sensing results – if the app has not detected speechover a certain amount of time, it exponentially decreases the sampling rate. Howto adjust the sampling rate with adaptive sensing, or hierarchical activation, whileensuring that the events of interest are not missed is an open research question. InSociableSense, a mobile application that senses socialisation among users [43], alinear reward-inaction function is associated with the sensing cycle, and the sam-pling rate is reduced during “quiet” times, when no interesting events are observed.The approach is very efficient with human interaction inference, since the targetevents, such as conversations, are not sudden and short; for other types of events,different approaches might be more appropriate.Client-server architecture is at the core of almost every ESM framework. Serversare used for centralised data storage and analysis, data visualisation, and remoteconfiguration of sampling mobiles. The balance of functionalities over mobiles anda server can have a significant impact on the performance and the possibilities of anmESM application. For automated labelling of human behaviour, say recognising ifa person is walking or not, a substantial amount of data, in this case accelerometerdata, has to be processed through machine learning models. Server-based processingcomes with benefits of a high computational power of multicore CPUs, and a globalview of the system, and as a consequence data from all the users can be harnessedfor individual (and group) inferences. On the other hand, the transfer of the high-volume data produced by mobile sensors can be costly, especially if done via acellular network. Some mESM and mobile sensing frameworks, including Ohmageand ESSensorManager, let developers define data transfer policies, such as “uploadmobile data only via Wi-Fi” and “do not upload any data if the battery level is below20%”.Besides its impact on performance and cost, balancing local and remote process-ing is important for ESM studies due to privacy issues associated with data transferand storage (see for example [9]). Location, video and audio data are particularlyvulnerable, yet can be protected with a suitable balance of remote and local process-ing. For example, if we want to infer that a user is having a conversation, insteadof transferring raw audio data for server-side processing, we can extract sound fea-tures relevant for speech classification, such as Mel-Frequency Cepstral Coefficients(MFCC) of sound frames that contain sounds over a certain threshold intensity, andsend these for remote analysis. Even if a malicious party gets access to this data, theoriginal audio recording cannot be reconstructed. Similarly, instead of sending rawgeographic coordinates to a server, a mobile application could host an internal clas-sifier of a user’s semantic location (e.g., home/work), and send already processeddata, minimising the amount of information about the user that can be revealed. obile-Based Experience Sampling for Behaviour Research 15
Mobile computing is rapidly transforming social sciences. First, the range of po-tential study participants has expanded dramatically. Nowadays researchers haveaccess to a virtually world-wide pool of participants. Second, the granularity of per-sonal data gathered through mobile sensors and phone-based interactions, includingonline social network activities, can paint a very detailed picture of an individual’sbehaviour. In addition, long-term data can be obtained, as long as the mESM appli-cation manages to keep users engaged, and they do not remove the application fromtheir phones.The above transformation requires the rethinking of the traditional social scienceapproaches. Computational social science [31] has emerged as a field that harnessesstatistical and machine learning approaches over user-generated “big data” in or-der to explain social science concepts, including behaviour. This field is inherentlyinterdisciplinary and rather broad, since it involves computer scientists, engineers,and social scientists, who traditionally had limited interactions in the past.
Ubiquitous mobile computing devices and the ESM provide a detailed assessmentof human behaviour at an unprecedented scale. A natural next step is to use theinformation about the existing individual and group behaviour to affect future be-haviour. Behaviour change interventions (BCIs) are a psychological method thataims to elicit a positive behaviour change. These interventions commonly includecollecting relevant information about the participant, setting goals and plans, mon-itoring behaviour, and providing feedback. Digital BCIs (dBCIs) moved behaviourchange interventions to the Web. The benefits of this transition include increasedcontrol over the content and the time of information delivery of information, as wellas the reduction in the intervention cost, since the need to a face-to-face interac-tion with a therapist is avoided. In addition, dBCIs open an opportunity for scalableautomatic content tailoring. Such tailoring has been shown to be effective for theactual behaviour change [52].Recently, both isolated [39] and systematic [28] attempts have been made tomove dBCIs from the Web to smartphones. The new platform enables interventioncontent delivery anywhere and anytime. In addition, a personalised use of the phoneindicates that through mobile sensing and user sampling a detailed personalisedmodel of user behaviour can be constructed and used to drive the intervention. Forexample, users whose samples indicate sedentary behaviour can be provided witha positive feedback whenever they are detected to be active. Technical difficultiesin building an integrated mESM and intervention distribution method hamper pro-liferation of mobile dBCIs. The system design and programming effort associatedwith implementing a system for remote mobile sensing and experience sampling, in-formation delivery, user management and personalised behaviour modelling is over- whelming. Certain existing frameworks, such as AndWellness [21] and BeWell [27],concentrate on sampling data relevant for users’ health and well-being, yet none ofthem cater specifically to dBCIs, and none of them solves the above technical dif-ficulties. The UBhave framework (Figure 2) aims to overcome this by providingout-of-the-box support for mobile dBCI design and deployment [18]. The frame-work consists of an intervention authoring tool, through which therapists can designinterventions, and an automated translation tool, which translates the design into anintervention file. This file is then deployed to and interpreted by participating mobiledevices running the UBhave client application. The framework ensures that thera-pist’s instructions on when to sample user experience and sensor data are followedby the mobile app, and that the behaviour changing advice is delivered to the userwhen needed.
Fig. 2
Overview of the UBhave framework for mobile digital behaviour change interventions.
The UBhave framework is the first that extends the idea of mESM beyond be-haviour tracking to behaviour change. While the idea of mobile dBCIs sounds ex-tremely promising, only after a wider adoption and broader behaviour change stud-ies it is will be possible to quantify the actual effectiveness of mobile interventions.
The awareness of the current context is the main novelty of experience samplingusing mobile devices. A prediction of future context has a tremendous potentialto make an mESM a key tool for explaining human behaviour. Although predict-ing, and even inferring, participants’ internal states with mobile sensors is yet to beachieved, prediction of some other behavioural aspects, such as users’ movementtrajectories or calling patterns, has already been demonstrated [42, 48]. obile-Based Experience Sampling for Behaviour Research 17
Anticipatory computing systems rely on the past, present and predicted futureinformation to bring judicious decisions about their current actions [47]. MobileESM applications could, in an anticipatory computing system manner, intelligentlyadapt their sampling schedules based on the predicted user behaviour. For example,an mESM application that could anticipate a depressive episode, could adapt itssampling to capture, with fine resolution, behaviour and the context just before theevent of interest. This would not only provide very detailed information about thecontext that lead to the onset of a depressive episode, but also use phone’s batteryresources more efficiently.Finally, we also envision proactive digital behaviour interventions delivered viamobile devices [40]. Besides the sampling schedule, anticipatory dBCIs would alsoadapt the feedback they give to a user according to the predicted state of the user,and the predicted effect the feedback will have on the user. For example, a smartwristband occasionally samples a user’s heart rate. Based on the readings, the sys-tem, encompassing a phone and a wristband, predicts that the user is in risk of beinghighly stressed out. The system accesses user’s online calendar and examines tasksscheduled for today. Then, it intelligently reschedules tasks to alleviate the risk ofhigh stress and suggests a new schedule to the user. Technical obstacles associatedwith this scenario include stress prediction, itself quite challenging, but also the pre-diction of how a user will react to a given change in the calendar. Will the changereally help alleviate stress? The idea of anticipatory mobile computing has just re-cently appeared in the literature, while the conventional mobile dBCIs have not yettaken off. Therefore, we are yet to witness anticipatory mobile dBCIs in practice.
Smarthphones and other mobile devices, such as wearables, are the first sensing andcomputing devices tightly interwoven into our daily lives. They represent revolu-tionary platforms for social science research and for the emerging field of computa-tional social science. They indeed open a window of opportunity for social scientiststo learn about human behaviour at previously unimaginable granularity and scale. Awide span of behaviours can be captured via mobile experience sampling. This cov-ers the domains for which the traditional experience sampling has already been em-ployed, such as studies of a personal time usage [25], emotions of different stigma-tised groups [11], and classroom activities [51], to name a few. In addition, mobilecomputing enables the investigation of new domains such as the location-dependentprivacy management of sharing information on online social networks [2], sleepmonitoring [27] and mobile application evaluation [6]. Furthermore, mESMs arebeing used for ambulatory assessment in areas that span from sexual behaviour tophysical exercise monitoring . The following URL lists currently running experience sampling projects using the Ohmageframework: http://ohmage.org/projects.html8 Veljko Pejovic, Neal Lathia, Cecilia Mascolo and Mirco Musolesi
We highlighted key benefits of mobile experience sampling, and presented mESMframeworks that abstract the technical effort of building a sensing and sampling mo-bile application, and enable seamless implementation and deployment of large scalesocial studies. Our vision for the evolution of mESM goes along the lines of thegeneral consensus that mobile applications need to be “stealth”, perfectly integratedwith everyday lives. Therefore, we envision considerate mESM studies, where theinteraction with the user is minimally invasive, and aligned with the sensed user be-haviour. Furthermore, harnessing the persuasive power of the smartphone, we alsosee proactive behaviour change interventions based on the automated analysis of thesampling results. Finally, outside of the scope of this review, but important for theecosystem of users, intervention designers and mESM framework developers are thequestions of large-scale data mining, interpretation and visualisation, data feedbackto study participants, and privacy and ethics issues associated with mobile sensing.
Acknowledgements
This work was supported through the EPSRC grants “UBhave: ubiquitousand social computing for positive behaviour change” (EP/I032673/1) and “Trajectories of Depres-sion: Investigating the Correlation between Human Mobility Patterns and Mental Health Problemsby means of Smartphones” (P/L006340/1).
References
1. Google’s Activity Recognition Application. http://developer.android.com/training/location/activity-recognition.html.2. F. B. Abdesslem, I. Parris, and T. Henderson. Mobile experience sampling: Reaching the partsof facebook other methods cannot reach. In
Proceedings of the Privacy and Usability MethodsPow-Wow (PUMP) , Dundee, UK, September 2010.3. N. Aharony, W. Pan, C. Ip, I. Khayal, and A. Pentland. Social fmri: Investigating and shapingsocial mechanisms in the real world.
Pervasive and Mobile Computing , 7(6):643–659, 2011.4. L. F. Barrett and D. J. Barrett. An introduction to computerized experience sampling in psy-chology.
Social Science Computer Review , 19(2):175–185, 2001.5. L. Clark and D. Watson. Mood and the Mundane: Relations Between Daily Life and Self-Reported Mood.
Journal of Personality and Social Psychology , 54(2):296–308, 1988.6. S. Consolvo and M. Walker. Using the experience sampling method to evaluate ubicompapplications.
IEEE Pervasive Computing , 2:24–31, 2003.7. S. Cote and D. S. Moskowitz. On the Dynamic Covariation Between Interpersonal Behav-ior and Affect: Prediction from Neuroticism, Extraversion, and Agreeableness.
Journal ofPersonality and Social Psychology , 75:1032, 1998.8. M. Csikszentmihalyi and R. Larson. Validity and reliability of the experience-samplingmethod.
The Journal of Nervous and Mental Disease , 175(9):526–536, 1987.9. Y.-A. de Montjoye, E. Shmueli, S. S. Wang, and A. S. Pentland. openPDS: Protecting thePrivacy of Metadata through SafeAnswers.
PloS one , 9(7):e98790, 2014.10. J. Fahrenberg and M. Myrtek, editors.
Ambulatory assessment: Computer- assisted psycho-logical and psychophysiological methods in monitoring and field studies . Hogrefe & Huber,Seattle, WA, USA, 1996.11. D. Frable, L. Platt, and S. Hoey. Concealable stigmas and positive self-perceptions: feelingbetter around similar others.
Journal of personality and social psychology , 74(4):909–921,1998.12. V. Frias-Martinez and J. Virsesa. On The Relationship Between Socio-Economic Factors andCell Phone Usage. In
ICTD’12 , Atlanta, GA, USA, March 2012.obile-Based Experience Sampling for Behaviour Research 1913. J. Froehlich, M. Y. Chen, S. Consolvo, B. Harrison, and J. A. Landay. MyExperience: ASystem for In Situ Tracing and Capturing of User Feedback on Mobile Phones. In
MobiSys’07 ,Puerto Rico, USA, June 2007.14. J. Froehlich, T. Dillahunt, P. Klasnja, J. Mankoff, S. Consolvo, B. Harrison, and J. A. Landay.UbiGreen: Investigating a Mobile Tool for Tracking and Supporting Green TransportationHabits. In
ACM CHI’09 , Boston, MA, USA, April 2009.15. A. Gaggioli, G. Pioggia, G. Tartarisco, G. Baldus, D. Corda, P. Cipresso, and G. Riva. A mo-bile data collection platform for mental health research.
Personal and Ubiquitous Computing
American Psychol-ogist , 59(2), 2004.18. C. Hargood, V. Pejovic, L. Morrison, D. T. Michaelides, M. Musolesi, L. Yardley, andM. Weal. The UBhave Framework: Dynamic Pervasive Applications for Behavioural Psy-chology. In
Mobiquitous’14 (poster session) , London, UK, December 2014.19. C. Hartung, A. Lerer, Y. Anokwa, C. Tseng, W. Brunette, and G. Borriello. Open data kit:Tools to build information services for developing regions. In
ACM DEV’10 , London, UK,December 2010.20. J. M. Hektner, J. A. Schmidt, and M. Csikszentmihalyi.
Experience Sampling Method: Mea-suring the Quality of Everyday Life . Sage Publications, Thousand Oaks, CA, USA, 2006.21. J. Hicks, N. Ramanathan, D. Kim, M. Monibi, J. Selsky, M. Hansen, and D. Estrin. AndWell-ness: An Open Mobile System for Activity and Experience Sampling. In
WirelessHealth’10 ,La Jolla, CA, USA, October 2010.22. J. Ho and S. S. Intille. Using Context-Aware Computing to Reduce the Perceived Burden ofInterruptions from Mobile Devices. In
CHI’05 , Portland, OR, USA, April 2005.23. S. S. Intille, J. Rondoni, C. Kukla, I. Ancona, and L. Bao. A Context-Aware ExperienceSampling Tool. In
CHI’03 (Extended Abstracts) , Lauderdale, FL, USA, April 2003.24. A. Kapoor and E. Horvitz. Experience Sampling for Building Predictive User Models: AComparative Study. In
CHI’08 , Florence, Italy, April 2008.25. R. Kubey and M. Csikszentmihalyi.
Television and the quality of life: How viewing shapeseveryday experience . Routledge, 1990.26. N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. T. Campbell. A Survey ofMobile Phone Sensing.
IEEE Communications Magazine , 48:140–150, 2010.27. N. D. Lane, M. Mohammod, M. Lin, X. Yang, H. Lu, S. Ali, A. Doryab, E. Berke, T. Choud-hury, and A. Campbell. Bewell: A smartphone application to monitor, model and promotewellbeing. In
PervasiveHealth’11 , pages 23–26, 2011.28. N. Lathia, V. Pejovic, K. Rachuri, C. Mascolo, M. Musolesi, and P. J. Rentfrow. Smartphonesfor Large-Scale Behaviour Change Intervention.
IEEE Pervasive Computing , 12(3), 2013.29. N. Lathia, K. Rachuri, C. Mascolo, and P. Rentfrow. Contextual Dissonance: Design Bias inSensor-Based Experience Sampling Methods. In
UbiComp’13 , Zurich, Switzerland, Septem-ber 2013.30. N. Lathia, K. K. Rachuri, C. Mascolo, and G. Roussos. Open Source Smartphone Librariesfor Computational Social Science. In
MCSS’13 , Zurich, Switzerland, September 2013.31. D. Lazer, A. Pentland, L. Adamic, S. Aral, A. Barabasi, D. Brewer, N. Christakis, N. Contrac-tor, J. Fowler, M. Gutmann, et al. Life in the network: the coming age of computational socialscience.
Science , 323(5915):721, 2009.32. H. Lu, D. Frauendorfer, M. Rabbi, M. S. Mast, G. T. Chittaranjan, A. T. Campbell, D. Gatica-Perez, and T. Choudhury. Stresssense: Detecting stress in unconstrained acoustic environ-ments using smartphones. In
UbiComp’12 , pages 351–360. ACM, 2012.33. W. Mason and S. Suri. Conducting Behavioral Research on Amazons Mechanical Turk.
Be-havior Research Methods , 44(1):1–23, 2012.34. M. R. Mehl and T. S. Conner, editors.
Handbook of Research Methods for Studying DailyLife . Guilford Publications, 2013.0 Veljko Pejovic, Neal Lathia, Cecilia Mascolo and Mirco Musolesi35. A. Mehrotra, V. Pejovic, and M. Musolesi. SenSocial: A Middleware for Integrating OnlineSocial Networks and Mobile Sensing Data Streams. In
Middleware’14 , Bordeaux, France,December 2014.36. A. Mehrotra, J. Vermeulen, V. Pejovic, and M. Musolesi. Ask, But Don’t Interrupt: The Casefor Interruptibility-Aware Mobile Experience Sampling. In , 2015.37. G. Miller. The Smartphone Psychology Manifesto.
Perspectives on Psychological Science ,7(3):221–237, 2012.38. E. Miluzzo, N. D. Lane, K. Fodor, R. Peterson, H. Lu, M. Musolesi, S. B. Eisenman, X. Zheng,and A. T. Campbell. Sensing meets mobile social networks: the design, implementation andevaluation of the cenceme application. In
SenSys’08 , pages 337–350. ACM, 2008.39. L. G. Morrison, C. Hargood, S. X. Lin, L. Dennison, J. Joseph, S. Hughes, D. T. Michaelides,D. Johnston, M. Johnston, S. Michie, et al. Understanding usage of a hybrid website andsmartphone app for weight management: A mixed-methods study.
Journal of Medical InternetResearch , 16(10), 2014.40. V. Pejovic and M. Musolesi. Anticipatory Mobile Computing for Behaviour Change Inter-ventions. In , Seattle, WA, USA, 2014.41. V. Pejovic and M. Musolesi. InterruptMe: Designing Intelligent Prompting Mechanisms forPervasive Applications. In
UbiComp’14 , Seattle, WA, USA, September 2014.42. S. Phithakkitnukoon, R. Dantu, R. Claxton, and N. Eagle. Behavior-Based Adaptive CallPredictor.
ACM Transactions on Autonomous and Adaptive Systems , 6(3):21, 2011.43. K. K. Rachuri, C. Mascolo, M. Musolesi, and P. J. Rentfrow. SociableSense: Exploring theTrade-offs of Adaptive Sampling and Computation Offloading for Social Sensing. In
Mobi-Com’11 , Las Vegas, NV, USA, September 2011.44. K. K. Rachuri, M. Musolesi, C. Mascolo, J. Rentfrow, C. Longworth, and A. Aucinas. Emo-tionSense: A Mobile Phones based Adaptive Platform for Experimental Social PsychologyResearch. In
UbiComp’10 , Copenhagen, Denmark, September 2010. ACM.45. N. Ramanathan, F. Alquaddoomi, H. Falaki, D. George, C. Hsieh, J. Jenkins, C. Ketcham,B. Longstaff, J. Ooms, J. Selsky, H. Tangmunarunkit, and D. Estrin. Ohmage: An OpenMobile System for Activity and Experience Sampling. In
PervasiveHealth’12 , San Diego,CA, USA, May 2012.46. H. T. Reis and S. L. Gable. Event-sampling and other methods for studying everyday experi-ence.
Handbook of Research Methods in Social and Personality Psychology , pages 190–222,2000.47. R. Rosen.
Anticipatory Systems . Pergamon Press, Oxford, UK, 1985.48. A. Sadilek and J. Krumm. Far Out: Predicting Long-Term Human Mobility. In
AAAI’12 ,Toronto, Canada, July 2012.49. G. H. Ter Hofte. Xensible interruptions from your mobile phone. In
Mobile HCI’07 , Singa-pore, September 2007.50. T. J. Trull and U. Ebner-Priemer. Ambulatory Assessment.
Annual Review of Clinical Psy-chology , 9(151), 2013.51. J. C. Turner, D. K. Meyer, K. E. Cox, C. Logan, M. DiCintio, and C. T. Thomas. Creat-ing contexts for involvement in mathematics.
Journal of Educational Psychology , 90(4):730,1998.52. L. Yardley, J. Joseph, S. Michie, M. Weal, G. Wills, and P. Little. Evaluation of a web-basedintervention providing tailored advice for self-management of minor respiratory symptoms:exploratory randomized controlled trial.