Roanna Lun

A Survey of Applications and Human Motion Recognition with Microsoft Kinect

Microsoft Kinect, a low-cost motion sensing device, enables users to interact with computers or game consoles naturally through gestures and spoken commands without any other peripheral equipment. As such, it has commanded intense interests in research and development on the Kinect technology. In this paper, we present, a comprehensive survey on Kinect applications, and the latest research and development on motion recognition using data captured by the Kinect sensor. On the applications front, we review the applications of the Kinect technology in a variety of areas, including healthcare, education and performing arts, robotics, sign language recognition, retail services, workplace safety training, as well as 3D reconstructions. On the technology front, we provide an overview of the main features of both versions of the Kinect sensor together with the depth sensing technologies used, and review literatures on human motion recognition techniques used in Kinect applications. We provide a classification of motion recognition techniques to highlight the different approaches used in human motion recognition. Furthermore, we compile a list of publicly available Kinect datasets. These datasets are valuable resources for researchers to investigate better methods for human motion recognition and lower-level computer vision tasks such as segmentation, object detection and human pose estimation.

Realtime Motion Assessment For Rehabilitation Exercises: Integration Of Kinematic Modeling With Fuzzy Inference

Abstract This article describes a novel approach to realtime motion assessment for rehabilitation exercises based on the integration of comprehensive kinematic modeling with fuzzy inference. To facilitate the assessment of all important aspects of a rehabilitation exercise, a kinematic model is developed to capture the essential requirements for static poses, dynamic movements, as well as the invariance that must be observed during an exercise. The kinematic model is expressed in terms of a set of kinematic rules. During the actual execution of a rehabilitation exercise, the similarity between the measured motion data and the model is computed in terms of their distances, which are then used as inputs to a fuzzy interference system to derive the overall quality of the execution. The integrated approach provides both a detailed categorical assessment of the overall execution of the exercise and the degree of adherence to individual kinematic rules.

A Kinect-based rehabilitation exercise monitoring and guidance system

In this paper, we describe the design and implementation of a Kinect-based system for rehabilitation exercises monitoring and guidance. We choose to use the Unity framework to implement our system because it enables us to use virtual reality techniques to demonstrate detailed movements to the patient, and to facilitate examination of the quality and quantity of the patient sessions by the clinician. The avatar-based rendering of motion also preserves the privacy of the patients, which is essential for healthcare systems. The key contribution of our research is a rule-based approach to realtime exercise quality assessment and feedback. We developed a set of basic rule elements that can be used to express the correctness rules for common rehabilitation exercises.

Rule based realtime motion assessment for rehabilitation exercises

In this paper, we describe a rule based approach to realtime motion assessment of rehabilitation exercises. We use three types of rules to define each exercise: (1) dynamic rules, with each rule specifying a sequence of monotonic segments of the moving joint or body segment, (2) static rules for stationary joints or body segments, and (3) invariance rules that dictate the requirements of moving joints or body segments. A finite state machine based approach is used in dynamic rule specification and realtime assessment. In addition to the typical advantages of the rule based approach, such as realtime motion assessment with specific feedback, our approach has the following advantages: (1) increased reusability of the defined rules as well as the rule assessment engine facilitated by a set of generic rule elements; (2) increased customizability of the rules for each exercise enabled by the use of a set of generic rule elements and the use of extensible rule encoding method; and (3) increased robustness without relying on expensive statistical algorithms to tolerate motion sensing errors and subtle patient errors.

A Human-Centered Activity Tracking System: Toward a Healthier Workplace

Lost productivity from lower back injuries in workplaces costs billions of U.S. dollars per year. A significant fraction of such workplace injuries are the result of workers not following best practices. In this paper, we present the design, implementation, and evaluation of a novel computer-vision-based system that aims to increase the workers’ compliance to best practices. The system consists of inexpensive programmable depth sensors, wearable devices, and smart phones. The system is designed to track the activities of consented workers using the depth sensors, alert them discreetly on detection of noncompliant activities, and produce cumulative reports on their performance. Essentially, the system provides a valuable set of services for both workers and administrators toward a healthier and, therefore, more productive workplace. This study advances the state of the art in the following ways: 1) a set of mechanisms that enable nonintrusive privacy-aware selective tracking of consented workers in the presence of people that should not be tracked; 2) a single sign-on worker identification mechanism; 3) a method that provides realtime detection of noncompliant activities; and 4) a usability study that provides invaluable feedback regarding system design and deployment, as well as future areas of improvements.

A privacy-aware Kinect-based system for healthcare professionals

In this paper, we present a novel system for healthcare professionals to enhance their compliance with best practices and regulations using Microsoft Kinect sensors and smart watches while strictly protecting patient privacy. A core contribution of this study is a registration mechanism for a healthcare professional to explicitly give our system the permission to monitor his or her activities. Our system supports the use of multiple Kinect sensors for improved tracking accuracy and better coverage for large workplaces. Furthermore, we introduce a non-intrusive biometrics-based single sign-on mechanism to allow a user to register once for all Kinect sensors within each session. Finally, our system generates alerts reliably on detection of non-compliant activities and delivers the alerts discreetly to a consented healthcare professional via a designated smart watch according to his/her personal preference.

LiftingDoneRight: A Privacy-Aware Human Motion Tracking System for Healthcare Professionals

This article describes the design and implementation of LiftingDoneRight, a novel system for healthcare professionals to enhance their compliance with best practices and regulations regarding proper body mechanics for lifting and pulling activities. The system uses Microsoft Kinect to track the motion of consented users non-intrusively. The system relies on the use of a smartwatch to deliver an alert via vibration and text display whenever a wrong activity that violated the proper body mechanics has been detected. A core contribution of this study is a registration mechanism for a healthcare professional to explicitly give permission to the system to monitor his or her activities. Furthermore, a non-intrusive biometrics-based single sign-on mechanism is incorporated into the system to allow a user to be automatically identified for tracking as long as the user has manually registered with the system before. Finally, the system offers a number of configurations to accommodate different usability needs and privacy requirements.

Tracking the activities of daily lives: An integrated approach

The tracking of the activities of daily living (ADL) may have significant implications in healthcare because it would enable healthcare professionals to receive updates remotely regarding the functional status of post-injury and post-surgery patients, and people with disabilities and the elderly. If successful, technologies that enable ADL tracking could dramatically reduce the healthcare cost because people could stay at the comfort of their home instead of at the healthcare facilities. In this paper, we present the design and implementation of a system that integrates two modalities of human motion tracking: computer vision and inertial sensing. For the computer vision modality, we choose to use the Microsoft Kinect sensor due to its low-cost and excellent programming support. For the inertial sensing modality, we choose to use smart watches for the same reason. The Kinect sensor is responsible for indoor ADL tracking, while the smart-watch component is responsible for outdoor ADL tracking. The smart-watch component also facilitates user-identification with the Kinect sensor and users health related activities information with indoor computing component, as well delivers realtime feedbacks to the users.

A Kinect-based system for promoting healthier living at home

In this paper, we present a novel system designed to promote healthy living at home. The system integrates Microsoft Kinect and wearable devices such as smart watches and fitness bands to enable selective tracking of user activities at the home setting. The objective of the system is to continuously monitor each user and detect bad postures that could increase the risk of back injuries, and prolonged sedentary bouts that are not conducive for a healthy lifestyle. The wearable device to be worn by each user also delivers realtime feedback to the user on detection of bad postures or inactivities. Furthermore, activities data are logged for each individual at a home server and can be assessed via mobile devices or regular Web browsers.

The design and implementation of a Kinect-based framework for selective human activity tracking

In this paper, we present the design and implementation details of a Kinect-based framework for human activity tracking. The framework is intentionally designed to be open so that it can communicate over the network with other systems and mobile/wearable devices. The possibility of integrating with other devices and systems makes it possible to use Kinect for human activity tracking in a way unforeseen before. For example, the integration of our framework with wearable sensors, such as smart watches and fitness bands, enables us to perform selective tracking of the daily activities of a particular user and provide realtime feedback to the user. Furthermore, multiple frameworks could work together to form a federated system to cover a large area and/or a large number of users.