Feiyan Hu

Periodicity detection in lifelog data with missing and irregularly sampled data

Lifelogging is the ambient, continuous digital recording of a persons everyday activities for a variety of possible applications. Much of the work to date in lifelogging has focused on developing sensors, capturing information, processing it into events and then supporting event-based access to the lifelog for applications like memory recall, behaviour analysis or similar. With the recent arrival of aggregating platforms such as Apples HealthKit, Microsofts HealthVault and Googles Fit, we are now able to collect and aggregate data from lifelog sensors, to centralize the management of data and in particular to search for and detect patterns of usage for individuals and across populations. In this paper, we present a framework that detects both low-level and high-level periodicity in lifelog data, detecting hidden patterns of which users would not otherwise be aware. We detect periodicities of time series using a combination of correlograms and periodograms, using various signal processing algorithms. Periodicity detection in lifelogs is particularly challenging because the lifelog data itself is not always continuous and can have gaps as users may use their lifelog devices intermittingly. To illustrate that periodicity can be detected from such data, we apply periodicity detection on three lifelog datasets with varying levels of completeness and accuracy.

Behavioral Periodicity Detection from 24 h Wrist Accelerometry and Associations with Cardiometabolic Risk and Health-Related Quality of Life

Periodicities (repeating patterns) are observed in many human behaviors. Their strength may capture untapped patterns that incorporate sleep, sedentary, and active behaviors into a single metric indicative of better health. We present a framework to detect periodicities from longitudinal wrist-worn accelerometry data. GENEActiv accelerometer data were collected from 20 participants (17 men, 3 women, aged 35–65) continuously for 64.4 ± 26.2 (range: 13.9 to 102.0) consecutive days. Cardiometabolic risk biomarkers and health-related quality of life metrics were assessed at baseline. Periodograms were constructed to determine patterns emergent from the accelerometer data. Periodicity strength was calculated using circular autocorrelations for time-lagged windows. The most notable periodicity was at 24 h, indicating a circadian rest-activity cycle; however, its strength varied significantly across participants. Periodicity strength was most consistently associated with LDL-cholesterol (rs = 0.40–0.79, Ps < 0.05) and triglycerides (rs = 0.68–0.86, Ps < 0.05) but also associated with hs-CRP and health-related quality of life, even after adjusting for demographics and self-rated physical activity and insomnia symptoms. Our framework demonstrates a new method for characterizing behavior patterns longitudinally which captures relationships between 24 h accelerometry data and health outcomes.

Using periodicity intensity to detect long term behaviour change

This paper introduces a new way to analyse and visualize quantified-self or lifelog data captured from any lifelogging device over an extended period of time. The mechanism works on the raw, unstructured lifelog data by detecting periodicities, those repeating patters that occur within our lifestyles at different frequencies including daily, weekly, seasonal, etc. Focusing on the 24 hour cycle, we calculate the strength of the 24-hour periodicity at 24-hour intervals over an extended period of a lifelog. Changes in this strength of the 24-hour cycle can illustrate changes or shifts in underlying human behavior. We have performed this analysis on several lifelog datasets of durations from several weeks to almost a decade, from recordings of training distances to sleep data. In this paper we use 24 hour accelerometer data to illustrate the technique, showing how changes in human behavior can be identified.

Visibility of Wearable Sensors as Measured Using Eye Tracking Glasses

Sensor technologies can enable independent living for people with dementia by monitoring their behaviour and identifying points where support may be required. Wearable sensors can provide such support but may constitute a source of stigma for the user if they are perceived as visible and therefore obtrusive. This paper presents an initial empirical investigation exploring the extent to which wearable sensors are perceived as visible. 23 Participants wore eye tracking glasses, which superimposed the location of their gaze onto video data of their panorama. Participants were led to believe that the research entailed a subjective evaluation of the eye tracking glasses. A researcher wore one of two wearable sensors during the evaluation enabling us to measure the extent to which participants fixated on the sensor during a one-on-one meeting. Results are presented on the general visibility and potential fixations on two wearable sensors, a wrist-work actigraph and a lifelogging camera, during normal conversation between two people. Further investigation is merited according to the results of this pilot study.

Formulating Queries for Collecting Training Examples in Visual Concept Classification

Video content can be automatically analysed and indexed using trained classifiers which map low-level features to semantic concepts. Such classifiers need training data consisting of sets of images which contain such concepts and recently it has been discovered that such training data can be located using text-based search to image databases on the internet. Formulating the text queries which locate these training images is the challenge we address here. In this paper we present preliminary results on TRECVid data of concept classification using automatically crawled images as training data and we compare the results with those obtained from manually annotated training sets.

Image Aesthetics and Content in Selecting Memorable Keyframes from Lifelogs

Visual lifelogging using wearable cameras accumulates large amounts of image data. To make them useful they are typically structured into events corresponding to episodes which occur during the wearer’s day. These events can be represented as a visual storyboard, a collection of chronologically ordered images which summarise the day’s happenings. In previous work, little attention has been paid to how to select the representative keyframes for a lifelogged event, apart from the fact that the image should be of good quality in terms of absence of blurring, motion artifacts, etc. In this paper we look at image aesthetics as a characteristic of wearable camera images. We show how this can be used in combination with content analysis and temporal offsets, to offer new ways for automatically selecting wearable camera keyframes. In this paper we implement several variations of the keyframe selection method and illustrate how it works using a publicly-available lifelog dataset.

Periodicity intensity for indicating behaviour shifts from lifelog data

Periodic phenomena or oscillating signals can be found frequently in nature and recent research has observed periodicity appearing in lifelog data, the automatic digital recording of everyday activities. In this paper we are exploring periodicity and intensity of periodicity in big data settings, especially when the data is noisy, unevenly sampled and incomplete. An interesting possibility is to compute the intensity or strength of detected periodicity across the time span of a lifelog to see if it reveals changes in this strength at different times, indicating shifts in underlying behaviour. In this paper we propose several metrics to estimate the intensity of periodicity, longitudinally. Evaluation of these metrics is conducted on simulated high-level activity data generated from a proposed model. We also explore periodicity intensity calculated from two real lifelog datasets using. One is “big” data consists of low-level accelerometer data and another one is high level athletic performance data.