Marjorie Skubic

Fall Detection in Homes of Older Adults Using the Microsoft Kinect

A method for detecting falls in the homes of older adults using the Microsoft Kinect and a two-stage fall detection system is presented. The first stage of the detection system characterizes a persons vertical state in individual depth image frames, and then segments on ground events from the vertical state time series obtained by tracking the person over time. The second stage uses an ensemble of decision trees to compute a confidence that a fall preceded on a ground event. Evaluation was conducted in the actual homes of older adults, using a combined nine years of continuous data collected in 13 apartments. The dataset includes 454 falls, 445 falls performed by trained stunt actors and nine naturally occurring resident falls. The extensive data collection allows for characterization of system performance under real-world conditions to a degree that has not been shown in other studies. Cross validation results are included for standing, sitting, and lying down positions, near (within 4 m) versus far fall locations, and occluded versus not occluded fallers. The method is compared against five state-of-the-art fall detection algorithms and significantly better results are achieved.

A smart home application to eldercare: Current status and lessons learned

To address an aging population, we have been investigating sensor networks for monitoring older adults in their homes. In this paper, we report ongoing work in which passive sensor networks have been installed in 17 apartments in an aging in place eldercare facility. The network under development includes simple motion sensors, video sensors, and a bed sensor that captures sleep restlessness and pulse and respiration levels. Data collection has been ongoing for over two years in some apartments. This longevity in sensor data collection is allowing us to study the data and develop algorithms for identifying alert conditions such as falls, as well as extracting typical daily activity patterns for an individual. The goal is to capture patterns representing physical and cognitive health conditions and then recognize when activity patterns begin to deviate from the norm. In doing so, we strive to provide early detection of potential problems which may lead to serious health events if left unattended. We describe the components of the network and show examples of logged sensor data with correlated references to health events. A summary is also included on the challenges encountered and the lessons learned as a result of our experiences in monitoring aging adults in their homes.

Linguistic summarization of video for fall detection using voxel person and fuzzy logic

In this paper, we present a method for recognizing human activity from linguistic summarizations of temporal fuzzy inference curves representing the states of a three-dimensional object called voxel person. A hierarchy of fuzzy logic is used, where the output from each level is summarized and fed into the next level. We present a two level model for fall detection. The first level infers the states of the person at each image. The second level operates on linguistic summarizations of voxel persons states and inference regarding activity is performed. The rules used for fall detection were designed under the supervision of nurses to ensure that they reflect the manner in which elders perform these activities. The proposed framework is extremely flexible. Rules can be modified, added, or removed, allowing for per-resident customization based on knowledge about their cognitive and physical ability.

Recognizing Falls from Silhouettes

A major problem among the elderly involves falling. The recognition of falls from video first requires the segmentation of the individual from the background. To ensure privacy, segmentation should result in a silhouette that is a binary map indicating only the body position of the individual in an image. We have previously demonstrated a segmentation method based on color that can recognize the silhouette and detect and remove shadows. After the silhouettes are obtained, we extract features and train hidden Markov models to recognize future performances of these known activities. In this paper, we present preliminary results that demonstrate the usefulness of this approach for distinguishing between a few common activities, specifically with fall detection in mind

Histogram of oriented normal vectors for object recognition with a depth sensor

We propose a feature, the Histogram of Oriented Normal Vectors (HONV), designed specifically to capture local geometric characteristics for object recognition with a depth sensor. Through our derivation, the normal vector orientation represented as an ordered pair of azimuthal angle and zenith angle can be easily computed from the gradients of the depth image. We form the HONV as a concatenation of local histograms of azimuthal angle and zenith angle. Since the HONV is inherently the local distribution of the tangent plane orientation of an object surface, we use it as a feature for object detection/classification tasks. The object detection experiments on the standard RGB-D dataset [1] and a self-collected Chair-D dataset show that the HONV significantly outperforms traditional features such as HOG on the depth image and HOG on the intensity image, with an improvement of 11.6% in average precision. For object classification, the HONV achieved 5.0% improvement over state-of-the-art approaches.

An acoustic fall detector system that uses sound height information to reduce the false alarm rate

More than one third of about 38 million adults 65 and older fall each year in the United States. To address the above problem we propose to develop an acoustic fall detection system (FADE) that will automatically signal a fall to the monitoring caregiver. As opposed to many existent fall detection systems that require the monitored person to wear devices such as accelerometers or gyroscopes at all times, our system is completely unobtrusive by not requiring any wearable devices. To reduce the false alarm rate we employ an array of acoustic sensors to obtain sound source height information. The sound is considered a false alarm if it comes from a source located at a height higher than 2 feet. We tested our system in a pilot study that consisted of a set of 23 falls performed by a stunt actor during six sessions of about 15 minutes each (1.3 hours in total). The actor was previously trained by our nursing collaborators to fall like an elderly person. The use of height information reduced the false alarm hourly rate from 32 to 5 at a 100% fall detection rate.

Evaluation of an inexpensive depth camera for in-home gait assessment

An investigation of a new, inexpensive depth camera device, the Microsoft Kinect, for passive gait assessment in home environments is presented. In order to allow older adults to safely continue living in independent settings as they age, the ability to assess their risk of falling, along with detecting the early onset of illness and functional decline, is essential. Daily measurements of temporal and spatial gait parameters would greatly facilitate such an assessment. Ideally, these measurements would be obtained passively, in normal daily activity, without the need for wearable devices or expensive equipment. In this work, the use of the inexpensive Microsoft Kinect for obtaining measurements of temporal and spatial gait parameters is evaluated against an existing web-camera based system, along with a Vicon marker-based motion capture system for ground truth. Techniques for extracting gait parameters from the Kinect data are described, as well as the potential advantages of the Kinect over the web-camera system for passive, in-home gait assessment.

Passive in-home measurement of stride-to-stride gait variability comparing vision and Kinect sensing

We present an analysis of measuring stride-to-stride gait variability passively, in a home setting using two vision based monitoring techniques: anonymized video data from a system of two web-cameras, and depth imagery from a single Microsoft Kinect. Millions of older adults fall every year. The ability to assess the fall risk of elderly individuals is essential to allowing them to continue living safely in independent settings as they age. Studies have shown that measures of stride-to-stride gait variability are predictive of falls in older adults. For this analysis, a set of participants were asked to perform a number of short walks while being monitored by the two vision based systems, along with a marker based Vicon motion capture system for ground truth. Measures of stride-to-stride gait variability were computed using each of the systems and compared against those obtained from the Vicon.

Modeling Human Activity From Voxel Person Using Fuzzy Logic

As part of an interdisciplinary collaboration on elder-care monitoring, a sensor suite for the home has been augmented with video cameras. Multiple cameras are used to view the same environment and the world is quantized into nonoverlapping volume elements (voxels). Through the use of silhouettes, a privacy protected image representation of the human acquired from multiple cameras, a 3-D representation of the human is built in real time, called voxel person. Features are extracted from voxel person and fuzzy logic is used to reason about the membership degree of a predetermined number of states at each frame. Fuzzy logic enables human activity, which is inherently fuzzy and case-based, to be reliably modeled. Membership values provide the foundation for rejecting unknown activities, something that nearly all current approaches are insufficient in doing. We discuss temporal fuzzy confidence curves for the common elderly abnormal activity of falling. The automated system is also compared to a ground truth acquired by a human. The proposed soft computing activity analysis framework is extremely flexible. Rules can be modified, added, or removed, allowing per-resident customization based on knowledge about their cognitive and functionality ability. To the best of our knowledge, this is a new application of fuzzy logic in a novel approach to modeling and monitoring human activity, in particular, the well-being of an elderly resident, from video.

Unobtrusive, Continuous, In-Home Gait Measurement Using the Microsoft Kinect

A system for capturing habitual, in-home gait measurements using an environmentally mounted depth camera, the Microsoft Kinect, is presented. Previous work evaluating the use of the Kinect sensor for in-home gait measurement in a lab setting has shown the potential of this approach. In this paper, a single Kinect sensor and computer were deployed in the apartments of older adults in an independent living facility for the purpose of continuous, in-home gait measurement. In addition, a monthly fall risk assessment protocol was conducted for each resident by a clinician, which included traditional tools such as the timed up a go and habitual gait speed tests. A probabilistic methodology for generating automated gait estimates over time for the residents of the apartments from the Kinect data is described, along with results from the apartments as compared to two of the traditionally measured fall risk assessment tools. Potential applications and future work are discussed.