Niamh Caprani

Passively recognising human activities through lifelogging

Lifelogging is the process of automatically recording aspects of ones life in digital form. This includes visual lifelogging using wearable cameras such as the SenseCam and in recent years many interesting applications for this have emerged and are being actively researched. One of the most interesting of these, and possibly the most far-reaching, is using visual lifelogs as a memory prosthesis but there are also applications in job-specific activity recording, general lifestyle analysis and market analysis. In this work we describe a technique which allowed us to develop automatic classifiers for visual lifelogs to infer different lifestyle traits or characteristics. Their accuracy was validated on a set of 95k manually annotated images and through one-on-one interviews with those who gathered the images. These automatic classifiers were then applied to a collection of over 3million lifelog images collected by 33 individuals sporadically over a period of 3.5years. From this collection we present a number of anecdotal observations to demonstrate the future potential of lifelogging to capture human behaviour. These anecdotes include: the eating habits of office workers; to the amount of time researchers spend outdoors through the year; to the observation that retired people in our study appear to spend quite a bit of time indoors eating with friends. We believe this work demonstrates the potential of lifelogging techniques to assist behavioural scientists in future.

Experiences of aiding autobiographical memory using the sensecam

Human memory is a dynamic system that makes accessible certain memories of events based on a hierarchy of information, arguably driven by personal significance. Not all events are remembered, but those that are tend to be more psychologically relevant. In contrast, lifelogging is the process of automatically recording aspects of ones life in digital form without loss of information. In this article we share our experiences in designing computer-based solutions to assist people review their visual lifelogs and address this contrast. The technical basis for our work is automatically segmenting visual lifelogs into events, allowing event similarity and event importance to be computed, ideas that are motivated by cognitive science considerations of how human memory works and can be assisted. Our work has been based on visual lifelogs gathered by dozens of people, some of them with collections spanning multiple years. In this review article we summarize a series of studies that have led to the development of a browser that is based on human memory systems and discuss the inherent tension in storing large amounts of data but making the most relevant material the most accessible.

Touch Screens for the Older User

It has been 20 years since Ben Schneiderman predicted that there would be an increase in the use of touch screen applications yet it has been only in recent years that this prediction has come to pass. The concept of a touch screen computer was first introduced in 1965 by E.A. Johnson who described the possibilities of touch screen technology to support air traffic controllers. In this article Johnson describes how “the touch display coupled to a computer can be considered as a keyboard”, a novel approach at the time. Touch screens were brought into the public domain in 1971 by Elographics, Inc. which instigated the development of public touch screen technology such as automated teller machines (ATMs) and information kiosks (Brown et al., 2011). Another milestone in the history of touch technology was the introduction of the personal touch screen computer, HP-150, developed by Hewlett-Packard in 1983 (Sukumar, 1984). The purpose of this early design was to offer individuals an intuitive technology option. Although touch screen systems have maintained this intuitiveness and ease of use over the years, problems that existed with early systems still provide challenges for designers and developers to this day.

The smartphone as a platform for wearable cameras in health research.

BACKGROUND The Microsoft SenseCam, a small camera that is worn on the chest via a lanyard, increasingly is being deployed in health research. However, the SenseCam and other wearable cameras are not yet in widespread use because of a variety of factors. It is proposed that the ubiquitous smartphones can provide a more accessible alternative to SenseCam and similar devices. PURPOSE To perform an initial evaluation of the potential of smartphones to become an alternative to a wearable camera such as the SenseCam. METHODS In 2012, adults were supplied with a smartphone, which they wore on a lanyard, that ran life-logging software. Participants wore the smartphone for up to 1 day and the resulting life-log data were both manually annotated and automatically analyzed for the presence of visual concepts. The results were compared to prior work using the SenseCam. RESULTS In total, 166,000 smartphone photos were gathered from 47 individuals, along with associated sensor readings. The average time spent wearing the device across all users was 5 hours 39 minutes (SD=4 hours 11 minutes). A subset of 36,698 photos was selected for manual annotation by five researchers. Software analysis of these photos supports the automatic identification of activities to a similar level of accuracy as for SenseCam images in a previous study. CONCLUSIONS Many aspects of the functionality of a SenseCam largely can be replicated, and in some cases enhanced, by the ubiquitous smartphone platform. This makes smartphones good candidates for a new generation of wearable sensing devices in health research, because of their widespread use across many populations. It is envisioned that smartphones will provide a compelling alternative to the dedicated SenseCam hardware for a number of users and application areas. This will be achieved by integrating new types of sensor data, leveraging the smartphones real-time connectivity and rich user interface, and providing support for a range of relatively sophisticated applications.

Designing a touch-screen sensecam browser to support an aging population

In this paper, we describe the HCI challenges associated with the novel domain of lifelogging for older users. The SenseCam is a passively capturing wearable camera, worn via a lanyard around the neck and used to create a personal lifelog or visual recording of the wearers life, which generates information that may be very helpful as a human memory aid. Indeed, given that memory defects are more marked in the elderly, we believe that lifelogging browsing techniques which are considerate of the elderly are imperative. Thus, the challenge tackled in this work was to design and integrate the lifelogging activity supported by new technologies in such a way that can easily be learned and used by older people, enabling them to enhance and enrich their lives with the new technologies. This work provides design practitioners of future lifelogging interfaces early sight of the lessons we have learned in making lifelogging technologies accessible to elderly non-computing literate participants.

Experiencing SenseCam: a case study interview exploring seven years living with a wearable camera

This paper presents the findings from an interview with CG, an individual who has worn an automated camera, the SenseCam, every day for the past seven years. Of interest to the study were the participants day-to-day experiences wearing the camera and whether these had changed since first wearing the camera. The findings presented outline the effect that wearing the camera has on his self-identity, relationships and interactions with people in the public. Issues relating to data capture, transfer and retrieval of lifelog images are also identified. These experiences inform us of the long-term effects of digital life capture and how lifelogging could progress in the future.

Browsing large personal multimedia archives in a lean-back environment

As personal digital archives of multimedia data become more ubiquitous, the challenge of supporting multimodal access to such archives becomes an important research topic. In this paper we present and positively evaluate a gesture-based interface to a personal media archive which operates on a living room TV using a Wiimote. We illustrate that Wiimote interaction can outperform a point-and-click interaction as reported in a user study. In addition, a set of guidelines is presented for organising and interacting with large personal media archives in the enjoyment oriented (lean-back) environment of the living room.

Does external walking environment affect gait patterns

The objective of this work is to develop an understanding of the relationship between mobility metrics obtained outside of the clinic or laboratory and the context of the external environment. Ten subjects walked with an inertial sensor on each shank and a wearable camera around their neck. They were taken on a thirty minute walk in which they mobilized over the following conditions; normal path, busy hallway, rough ground, blind folded and on a hill. Stride time, stride time variability, stance time and peak shank rotation rate during swing were calculated using previously published algorithms. Stride time was significantly different between several of the conditions. Technological advances mean that gait variables can now be captured as patients go about their daily lives. The results of this study show that the external environment has a significant impact on the quality of gait metrics. Thus, context of external walking environment is an important consideration when analyzing ambulatory gait metrics from the unsupervised home and community setting.

Life-Long Collections: Motivations and the Implications for Lifelogging with Mobile Devices

In this paper the authors investigate the motivations for life-long collections and how these motivations can inform the design of future lifelog systems. Lifelogging is the practice of automatically capturing data from daily life experiences with mobile devices, such as smartphones and wearable cameras. Lifelog archives can benefit both older and younger people; therefore lifelog systems should be designed for people of all ages. The authors believe that people would be more likely to adopt lifelog practices that support their current motivations for collecting items. To identify these motivations, ten older and ten younger participants were interviewed. It was found that motivations for and against life-long collections evolve as people age and enter different stages, and that family is at the core of life-long collections. These findings will be used to guide the design of an intergenerational lifelog browser.

Remember when: development of an interactive reminiscence device

THE use of technology has become increasingly widespread across the globe. Findings from a feasibility study for the proposed device suggests that a population of 45-65 year olds are experienced using technology. Therefore this experience will be carried through to old age facilitating the use of emerging technologies, such as a technology driven reminiscence device. Furthermore a case study revealed that stimulation may aid communication amongst elderly people. A technology based reminiscence device is proposed as a means of facilitating communication creating bonds among elderly people. This reminiscence device in turn should alleviate loneliness by initiating conversation between elderly people. The proposed device is targeted at elderly people living in a nursing home, elderly people living alone and as a tool for care workers.