Emma Berry

SenseCam: a retrospective memory aid

This paper presents a novel ubiquitous computing device, the SenseCam, a sensor augmented wearable stills camera. SenseCam is designed to capture a digital record of the wearers day, by recording a series of images and capturing a log of sensor data. We believe that reviewing this information will help the wearer recollect aspects of earlier experiences that have subsequently been forgotten, and thereby form a powerful retrospective memory aid. In this paper we review existing work on memory aids and conclude that there is scope for an improved device. We then report on the design of SenseCam in some detail for the first time. We explain the details of a first in-depth user study of this device, a 12-month clinical trial with a patient suffering from amnesia. The results of this initial evaluation are extremely promising; periodic review of images of events recorded by SenseCam results in significant recall of those events by the patient, which was previously impossible. We end the paper with a discussion of future work, including the application of SenseCam to a wider audience, such as those with neurodegenerative conditions such as Alzheimers disease.

The use of a wearable camera, SenseCam, as a pictorial diary to improve autobiographical memory in a patient with limbic encephalitis: A preliminary report

This case study describes the use of a wearable camera, SenseCam, which automatically captures several hundred images per day, to aid autobiographical memory in a patient, Mrs B, with severe memory impairment following limbic encephalitis. By using SenseCam to record personally experienced events we intended that SenseCam pictures would form a pictorial diary to cue and consolidate autobiographical memories. After wearing SenseCam, Mrs B plugged the camera into a PC which uploaded the recorded images and allowed them to be viewed at speed, like watching a movie. In the control condition, a written diary was used to record and remind her of autobiographical events. After viewing SenseCam images, Mrs B was able to recall approximately 80% of recent, personally experienced events. Retention of events was maintained in the long-term, 11 months afterwards, and without viewing SenseCam images for three months. After using the written diary, Mrs B was able to remember around 49% of an event; after one month with no diary readings she had no recall of the same events. We suggest that factors relating to rehearsal/re-consolidation may have enabled SenseCam images to improve Mrs Bs autobiographical recollection.

SenseCam: A wearable camera that stimulates and rehabilitates autobiographical memory

SenseCam is a wearable digital camera that captures an electronic record of the wearers day. It does this by automatically recording a series of still images through its wide-angle lens, and simultaneously capturing a log of data from a number of built-in electronic sensors. Subsequently reviewing a sequence of images appears to provide a powerful autobiographical memory cue. A preliminary evaluation of SenseCam with a patient diagnosed with severe memory impairment was extremely positive; periodic review of images of events recorded by SenseCam resulted in significant recall of those events. Following this, a great deal of work has been undertaken to explore this phenomenon and there are early indications that SenseCam technology may be beneficial to a variety of patients with physical and mental health problems, and is valuable as a tool for investigating normal memory through behavioural and neuroimaging means. Elsewhere, it is becoming clear that judicious use of SenseCam could significantly impact the study of human behaviour. Meanwhile, research and development of the technology itself continues with the aim of providing robust hardware and software tools to meet the needs of clinicians, patients, carers, and researchers. In this paper we describe the history of SenseCam, and the design and operation of the SenseCam device and the associated viewing software, and we discuss some of the ongoing research questions being addressed with the help of SenseCam.

Can we use digital life-log images to investigate active and sedentary travel behaviour? Results from a pilot study.

BackgroundActive travel such as walking and cycling has potential to increase physical activity levels in sedentary individuals. Motorised car travel is a sedentary behaviour that contributes to carbon emissions. There have been recent calls for technology that will improve our ability to measure these travel behaviours, and in particular evaluate modes and volumes of active versus sedentary travel. The purpose of this pilot study is to investigate the potential efficacy of a new electronic measurement device, a wearable digital camera called SenseCam, in travel research.MethodsParticipants (n = 20) were required to wear the SenseCam device for one full day of travel. The device automatically records approximately 3,600 time-stamped, first-person point-of-view images per day, without any action required by the wearer. Participants also completed a self-report travel diary over the same period for comparison, and were interviewed afterwards to assess user burden and experience.ResultsThere were a total of 105 confirmed journeys in this pilot. The new SenseCam device recorded more journeys than the travel diary (99 vs. 94). Although the two measures demonstrated an acceptable correlation for journey duration (r = 0.92, p < 0.001) self-reported journey duration was over-reported (mean difference 154 s per journey; 95% CI = 89 to 218 s; 95% limits of agreement = 154 ± 598 s (-444 to 752 s)). The device also provided visual data that was used for directed interviews about sources of error.ConclusionsDirect observation of travel behaviour from time-stamped images shows considerable potential in the field of travel research. Journey duration derived from direct observation of travel behaviour from time-stamped images appears to suggest over-reporting of self-reported journey duration.

Wearable cameras in health: the state of the art and future possibilities.

The relationships between lifestyle behaviors and health outcomes usually are based on self-reported data. Such data are prone to measurement error. In response, there has been a movement towards objective forms of measurement that have low participant and researcher burden. The papers in this theme issue in the American Journal of Preventive Medicine assess the utility of a new form of objective measurement in health research, namely wearable cameras. These devices can be worn all day and automatically record images from a first-person point of view, requiring no intervention or attention from the subject or the researcher. The most mature visual lifelogging device is Microsofts SenseCam, a wearable camera worn via a lanyard around the neck. The SenseCam has been increasingly used in health-related research for several years. These theme papers report current research into wearable cameras in health, as presented at the SenseCam 2012 Symposium. Wearable cameras and their associated software analysis tools have developed to the point that they now appear well suited to measure sedentary behaviour, active travel, and nutrition-related behaviours. Individuals may recall events more accurately after reviewing images from their wearable cameras. Aspects of their immediate cognitive functioning may also improve. Despite the benefits of wearable cameras, there are still challenges remaining before their use becomes widespread. Ethical and privacy concerns are important issues that need to be addressed, as well as easy access to devices. In response, an ethical framework and smartphone-based wearable camera capture platform are proposed. In sum, this body of work suggests that the use of wearable cameras will soon be appropriate to understand lifestyle behaviours and the context in which the occur.

Theme: Wearable cameras in healthWearable Cameras in Health: The State of the Art and Future Possibilities

The relationships between lifestyle behaviors and health outcomes usually are based on self-reported data. Such data are prone to measurement error. In response, there has been a movement towards objective forms of measurement that have low participant and researcher burden. The papers in this theme issue in the American Journal of Preventive Medicine assess the utility of a new form of objective measurement in health research, namely wearable cameras. These devices can be worn all day and automatically record images from a first-person point of view, requiring no intervention or attention from the subject or the researcher. The most mature visual lifelogging device is Microsofts SenseCam, a wearable camera worn via a lanyard around the neck. The SenseCam has been increasingly used in health-related research for several years. These theme papers report current research into wearable cameras in health, as presented at the SenseCam 2012 Symposium. Wearable cameras and their associated software analysis tools have developed to the point that they now appear well suited to measure sedentary behaviour, active travel, and nutrition-related behaviours. Individuals may recall events more accurately after reviewing images from their wearable cameras. Aspects of their immediate cognitive functioning may also improve. Despite the benefits of wearable cameras, there are still challenges remaining before their use becomes widespread. Ethical and privacy concerns are important issues that need to be addressed, as well as easy access to devices. In response, an ethical framework and smartphone-based wearable camera capture platform are proposed. In sum, this body of work suggests that the use of wearable cameras will soon be appropriate to understand lifestyle behaviours and the context in which the occur.

SenseCam improves memory for recent events and quality of life in a patient with memory retrieval difficulties.

A wearable camera that takes pictures automatically, SenseCam, was used to generate images for rehearsal, promoting consolidation and retrieval of memories for significant events in a patient with memory retrieval deficits. SenseCam images of recent events were systematically reviewed over a 2-week period. Memory for these events was assessed throughout and longer-term recall was tested up to 6 months later. A written diary control condition followed the same procedure. The SenseCam review procedure resulted in significantly more details of an event being recalled, with twice as many details recalled at 6 months follow up compared to the written diary method. Self-report measures suggested autobiographical recollection was triggered by the SenseCam condition but not by reviewing the written diary. Emotional and social wellbeing questionnaires indicated improved confidence and decreased anxiety as a result of memory rehearsal using SenseCam images. We propose that SenseCam images provide a powerful boost to autobiographical recall, with secondary benefits for quality of life.

The neural basis of effective memory therapy in a patient with limbic encephalitis

Background: An fMRI study is described in which a postencephalitic woman with amnesia used a wearable camera which takes photographs passively, without user intervention, to record and review recent autobiographical events. “SenseCam” generates hundreds of images which can subsequently be reviewed quickly or one by one. Results: Memory for a significant event was improved substantially when tested after 4.5 weeks, if the patient viewed SenseCam images of the event every 2 days for 3 weeks. In contrast, after only 3.5 weeks, her memory was at chance levels for a similarly significant event which was reviewed equally often, but using a written diary. During the fMRI scan, the patient viewed images of these two events, plus images of an unrehearsed event and images from a novel “control” event that she had never experienced. There was no difference in behavioural responses or in activation when the unrehearsed and novel conditions were compared. Relative to the written-rehearsed condition, successful recognition of the images in the SenseCam-rehearsed condition was associated with activation of frontal and posterior cortical regions associated with normal episodic memory. Conclusion: SenseCam images may provide powerful cues that trigger the recall and consolidation of stored but inaccessible memories.

An exploratory case study of the impact of ambient biographical displays on identity in a patient with Alzheimer's disease

One of the most troubling symptoms of Alzheimers disease is the loss of the patients sense of identity. This loss complicates relationships, increases apathy, and generally impedes quality of life for the patient. We describe a novel in-home ambient display called Biography Theatre that cycles through music, photographs, movies, and narratives drawn from the patients past and current life. We conducted an exploratory case study with an 84-year-old male with moderate-stage Alzheimers disease (Mr H). The study consisted of three phases: a baseline phase, a phase wherein autobiographical materials were collected and discussed, and a phase wherein the display was deployed in the home. The patient demonstrated improvement on standardised tests of apathy and positive self-identity, but did not improve on tests of autobiographical memory, anxiety, depression, and general cognition. We also report on caregiver reactions to the intervention and how the display helped them cope with and reinterpret their loved ones condition. This work suggests that interdisciplinary work involving “off the desktop” computing technologies may be a fruitful way to provide rehabilitative benefit for individuals with Alzheimers disease.

Executive dysfunction in a case of transoral-frontal self-inflicted gunshot injury

Objective: This study presents an in-depth neuropsychological study of the consequences of a survivable transoral penetrating gunshot injury resulting in lesions to frontal lobe structures. Design: A case study of a woman who sustained a self-inflicted transoral gunshot injury that resulted in focal pathology to the frontal lobes. Methods: Neuropsychological examination with a comprehensive battery of standard tests. Results: Striking contrasts in performance were found on tests of executive function relating to planning and problem-solving ability, with marked impairment on a problem-solving task involving an action plan. Intact performance was seen on tests of memory. Conclusions: This case study demonstrates that transoral gunshot injury to the brain is survivable, though selective executive dysfunction may remain.