Aiden R. Doherty

LifeLogging: Personal Big Data

We have recently observed a convergence of technologies to foster the emergence of lifelogging as a mainstream activity. Computer storage has become significantly cheaper, and advancements in sensing technology allows for the efficient sensing of personal activities, locations and the environment. This is best seen in the growing popularity of the quantified self movement, in which life activities are tracked using wearable sensors in the hope of better understanding human performance in a variety of tasks. This review aims to provide a comprehensive summary of lifelogging, to cover its research history, current technologies, and applications. Thus far, most of the lifelogging research has focused predominantly on visual lifelogging, hence we maintain this focus in this review. However, we also reflect on the challenges lifelogging poses for information access and retrieval in general. This review is a suitable reference for those seeking an information retrieval scientists perspective on lifelogging and the quantified self.

An Ethical Framework for Automated, Wearable Cameras in Health Behavior Research

Technologic advances mean automated, wearable cameras are now feasible for investigating health behaviors in a public health context. This paper attempts to identify and discuss the ethical implications of such research, in relation to existing guidelines for ethical research in traditional visual methodologies. Research using automated, wearable cameras can be very intrusive, generating unprecedented levels of image data, some of it potentially unflattering or unwanted. Participants and third parties they encounter may feel uncomfortable or that their privacy has been affected negatively. This paper attempts to formalize the protection of all according to best ethical principles through the development of an ethical framework. Respect for autonomy, through appropriate approaches to informed consent and adequate privacy and confidentiality controls, allows for ethical research, which has the potential to confer substantial benefits on the field of health behavior research.

Can we use digital life-log images to investigate active and sedentary travel behaviour? Results from a pilot study.

BackgroundActive travel such as walking and cycling has potential to increase physical activity levels in sedentary individuals. Motorised car travel is a sedentary behaviour that contributes to carbon emissions. There have been recent calls for technology that will improve our ability to measure these travel behaviours, and in particular evaluate modes and volumes of active versus sedentary travel. The purpose of this pilot study is to investigate the potential efficacy of a new electronic measurement device, a wearable digital camera called SenseCam, in travel research.MethodsParticipants (n = 20) were required to wear the SenseCam device for one full day of travel. The device automatically records approximately 3,600 time-stamped, first-person point-of-view images per day, without any action required by the wearer. Participants also completed a self-report travel diary over the same period for comparison, and were interviewed afterwards to assess user burden and experience.ResultsThere were a total of 105 confirmed journeys in this pilot. The new SenseCam device recorded more journeys than the travel diary (99 vs. 94). Although the two measures demonstrated an acceptable correlation for journey duration (r = 0.92, p < 0.001) self-reported journey duration was over-reported (mean difference 154 s per journey; 95% CI = 89 to 218 s; 95% limits of agreement = 154 ± 598 s (-444 to 752 s)). The device also provided visual data that was used for directed interviews about sources of error.ConclusionsDirect observation of travel behaviour from time-stamped images shows considerable potential in the field of travel research. Journey duration derived from direct observation of travel behaviour from time-stamped images appears to suggest over-reporting of self-reported journey duration.

Using the SenseCam to Improve Classifications of Sedentary Behavior in Free-Living Settings

BACKGROUND Studies have shown relationships between important health outcomes and sedentary behavior, independent of physical activity. There are known errors in tools employed to assess sedentary behavior. Studies of accelerometers have been limited to laboratory environments. PURPOSE To assess a broad range of sedentary behaviors in free-living adults using accelerometers and a Microsoft SenseCam that can provide an objective observation of sedentary behaviors through first person-view images. METHODS Participants were 40 university employees who wore a SenseCam and Actigraph accelerometer for 3-5 days. Images were coded for sitting and standing posture and 12 activity types. Data were merged and aggregated to a 60-second epoch. Accelerometer counts per minute (cpm) of <100 were compared with coded behaviors. Sensitivity and specificity analyses were performed. Data were collected in June and July 2011 and analyzed in April 2012. RESULTS TV viewing, other screen use, and administrative activities were correctly classified by the 100-cpm cutpoint. However, standing behaviors also fell under this threshold, and driving behaviors exceeded it. Multiple behaviors occurred simultaneously. A nearly 30-minute per day difference was found in sedentary behavior estimates based on the accelerometer versus the SenseCam. CONCLUSIONS Researchers should be aware of the strengths and weaknesses of the 100-cpm accelerometer cutpoint for identifying sedentary behavior. The SenseCam may be a useful tool in free-living conditions to better understand health behaviors such as sitting.

Theme: Wearable cameras in healthWearable Cameras in Health: The State of the Art and Future Possibilities

The relationships between lifestyle behaviors and health outcomes usually are based on self-reported data. Such data are prone to measurement error. In response, there has been a movement towards objective forms of measurement that have low participant and researcher burden. The papers in this theme issue in the American Journal of Preventive Medicine assess the utility of a new form of objective measurement in health research, namely wearable cameras. These devices can be worn all day and automatically record images from a first-person point of view, requiring no intervention or attention from the subject or the researcher. The most mature visual lifelogging device is Microsofts SenseCam, a wearable camera worn via a lanyard around the neck. The SenseCam has been increasingly used in health-related research for several years. These theme papers report current research into wearable cameras in health, as presented at the SenseCam 2012 Symposium. Wearable cameras and their associated software analysis tools have developed to the point that they now appear well suited to measure sedentary behaviour, active travel, and nutrition-related behaviours. Individuals may recall events more accurately after reviewing images from their wearable cameras. Aspects of their immediate cognitive functioning may also improve. Despite the benefits of wearable cameras, there are still challenges remaining before their use becomes widespread. Ethical and privacy concerns are important issues that need to be addressed, as well as easy access to devices. In response, an ethical framework and smartphone-based wearable camera capture platform are proposed. In sum, this body of work suggests that the use of wearable cameras will soon be appropriate to understand lifestyle behaviours and the context in which the occur.

Large scale population assessment of physical activity using wrist worn accelerometers: The UK Biobank Study

Background Physical activity has not been objectively measured in prospective cohorts with sufficiently large numbers to reliably detect associations with multiple health outcomes. Technological advances now make this possible. We describe the methods used to collect and analyse accelerometer measured physical activity in over 100,000 participants of the UK Biobank study, and report variation by age, sex, day, time of day, and season. Methods Participants were approached by email to wear a wrist-worn accelerometer for seven days that was posted to them. Physical activity information was extracted from 100Hz raw triaxial acceleration data after calibration, removal of gravity and sensor noise, and identification of wear / non-wear episodes. We report age- and sex-specific wear-time compliance and accelerometer measured physical activity, overall and by hour-of-day, week-weekend day and season. Results 103,712 datasets were received (44.8% response), with a median wear-time of 6.9 days (IQR:6.5–7.0). 96,600 participants (93.3%) provided valid data for physical activity analyses. Vector magnitude, a proxy for overall physical activity, was 7.5% (2.35mg) lower per decade of age (Cohen’s d = 0.9). Women had a higher vector magnitude than men, apart from those aged 45-54yrs. There were major differences in vector magnitude by time of day (d = 0.66). Vector magnitude differences between week and weekend days (d = 0.12 for men, d = 0.09 for women) and between seasons (d = 0.27 for men, d = 0.15 for women) were small. Conclusions It is feasible to collect and analyse objective physical activity data in large studies. The summary measure of overall physical activity is lower in older participants and age-related differences in activity are most prominent in the afternoon and evening. This work lays the foundation for studies of physical activity and its health consequences. Our summary variables are part of the UK Biobank dataset and can be used by researchers as exposures, confounding factors or outcome variables in future analyses.

Feasibility of a SenseCam-assisted 24-h recall to reduce under-reporting of energy intake.

Background/Objectives:The SenseCam is a camera worn on a lanyard around the neck that automatically captures point-of-view images in response to movement, heat and light (every 20–30 s). This device may enhance the accuracy of self-reported dietary intake by assisting participants’ recall of food and beverage consumption. It was the objective of this study to evaluate if the wearable camera, SenseCam, can enhance the 24-h dietary recall by providing visual prompts to improve recall of food and beverage consumption.Subject/Methods:Thirteen volunteer adults in Oxford, United Kingdom, were recruited. Participants wore the SenseCam for 2 days while continuing their usual daily activities. On day 3, participants’ diets were assessed using an interviewer-administered 24-h recall. SenseCam images were then shown to the participants and any additional dietary information that participants provided after viewing the images was recorded. Energy and macronutrient intakes were compared between the 24-h recall and 24-h recall+SenseCam.Results:Data from 10 participants were included in the final analysis (8 males and 2 females), mean age 33±11 years, mean BMI 25.9±5.1 kg/m2. Viewing the SenseCam images increased self-reported energy intake by approximately 1432±1564 kJ or 12.5% compared with the 24-h recall alone (P=0.02). The increase was predominantly due to reporting of 41 additional foods (241 vs 282 total foods) across a range of food groups. Eight changes in portion size were made, which resulted in a negligible change to energy intake.Conclusions:Wearable cameras are promising method to enhance the accuracy of self-reported dietary assessment methods.

Automatically augmenting lifelog events using pervasively generated content from millions of people.

In sensor research we take advantage of additional contextual sensor information to disambiguate potentially erroneous sensor readings or to make better informed decisions on a single sensor’s output. This use of additional information reinforces, validates, semantically enriches, and augments sensed data. Lifelog data is challenging to augment, as it tracks one’s life with many images including the places they go, making it non-trivial to find associated sources of information. We investigate realising the goal of pervasive user-generated content based on sensors, by augmenting passive visual lifelogs with “Web 2.0” content collected by millions of other individuals.

Evaluating Digital Health Interventions: Key Questions and Approaches

Digital health interventions have enormous potential as scalable tools to improve health and healthcare delivery by improving effectiveness, efficiency, accessibility, safety, and personalization. Achieving these improvements requires a cumulative knowledge base to inform development and deployment of digital health interventions. However, evaluations of digital health interventions present special challenges. This paper aims to examine these challenges and outline an evaluation strategy in terms of the research questions needed to appraise such interventions. As they are at the intersection of biomedical, behavioral, computing, and engineering research, methods drawn from all of these disciplines are required. Relevant research questions include defining the problem and the likely benefit of the digital health intervention, which in turn requires establishing the likely reach and uptake of the intervention, the causal model describing how the intervention will achieve its intended benefit, key components, and how they interact with one another, and estimating overall benefit in terms of effectiveness, cost effectiveness, and harms. Although RCTs are important for evaluation of effectiveness and cost effectiveness, they are best undertaken only when: (1) the intervention and its delivery package are stable; (2) these can be implemented with high fidelity; and (3) there is a reasonable likelihood that the overall benefits will be clinically meaningful (improved outcomes or equivalent outcomes at lower cost). Broadening the portfolio of research questions and evaluation methods will help with developing the necessary knowledge base to inform decisions on policy, practice, and research.

Evaluating the Feasibility of Measuring Travel to School Using a Wearable Camera

Background The school journey is often studied in relation to health outcomes in children and adolescents. Self-report is the most common measurement tool. Purpose To investigate the error on self-reported journey duration in adolescents, using a wearable digital camera (Microsoft SenseCam). Methods During March–May 2011, participants (n=17; aged 13–15 years) from four schools wore wearable cameras to and from school for 1 week. The device automatically records time-stamped, first-person point-of-view images, without any action from the wearer. Participants also completed a researcher-administered self-report travel survey over the same period. Analysis took place in November 2011. Within- and between-subjects correlation coefficients and Bland-Altman 95% limits of agreement were derived, accounting for the multiple observations per individual. Results Self-report data were collected for 150 journey stages and SenseCam data for 135 (90%) of these. The within-subjects correlation coefficient for journey duration was 0.89 (95% CI=0.84, 0.93). The between-subjects correlation coefficient was 0.92 (95% CI=0.79, 0.97). The mean difference (bias) between methods at the whole sample level was small (10 seconds per journey, 95% CI= −33, 53). The wide limits of agreement (±501 seconds, 95% CI= −491, 511) reveal large random error. Conclusions Compared to direct observation from images, self-reported journey duration is accurate at the mean group level but imprecise at the level of the individual participant.