José Ramón Padilla-López

Evolutionary joint selection to improve human action recognition with RGB-D devices

Interest in RGB-D devices is increasing due to their low price and the wide range of possible applications that come along. These devices provide a marker-less body pose estimation by means of skeletal data consisting of 3D positions of body joints. These can be further used for pose, gesture or action recognition. In this work, an evolutionary algorithm is used to determine the optimal subset of skeleton joints, taking into account the topological structure of the skeleton, in order to improve the final success rate. The proposed method has been validated using a state-of-the-art RGB action recognition approach, and applying it to the MSR-Action3D dataset. Results show that the proposed algorithm is able to significantly improve the initial recognition rate and to yield similar or better success rates than the state-of-the-art methods.

Fusion of Skeletal and Silhouette-Based Features for Human Action Recognition with RGB-D Devices

Since the Microsoft Kinect has been released, the usage of marker-less body pose estimation has been enormously eased. Based on 3D skeletal pose information, complex human gestures and actions can be recognised in real time. However, due to errors in tracking or occlusions, the obtained information can be noisy. Since the RGB-D data is available, the 3D or 2D shape of the person can be used instead. However, depending on the viewpoint and the action to recognise, it might present a low discriminative value. In this paper, the combination of body pose estimation and 2D shape, in order to provide additional characteristic value, is considered so as to improve human action recognition. Using efficient feature extraction techniques, skeletal and silhouette-based features are obtained which are low dimensional and can be obtained in real time. These two features are then combined by means of feature fusion. The proposed approach is validated using a state-of-the-art learning method and the MSR Action3D dataset as benchmark. The obtained results show that the fused feature achieves to improve the recognition rates, outperforming state-of-the-art results in recognition rate and robustness.

A vision-based system for intelligent monitoring: human behaviour analysis and privacy by context.

Due to progress and demographic change, society is facing a crucial challenge related to increased life expectancy and a higher number of people in situations of dependency. As a consequence, there exists a significant demand for support systems for personal autonomy. This article outlines the vision@home project, whose goal is to extend independent living at home for elderly and impaired people, providing care and safety services by means of vision-based monitoring. Different kinds of ambient-assisted living services are supported, from the detection of home accidents, to telecare services. In this contribution, the specification of the system is presented, and novel contributions are made regarding human behaviour analysis and privacy protection. By means of a multi-view setup of cameras, peoples behaviour is recognised based on human action recognition. For this purpose, a weighted feature fusion scheme is proposed to learn from multiple views. In order to protect the right to privacy of the inhabitants when a remote connection occurs, a privacy-by-context method is proposed. The experimental results of the behaviour recognition method show an outstanding performance, as well as support for multi-view scenarios and real-time execution, which are required in order to provide the proposed services.

Visual Privacy by Context: Proposal and Evaluation of a Level-Based Visualisation Scheme

Privacy in image and video data has become an important subject since cameras are being installed in an increasing number of public and private spaces. Specifically, in assisted living, intelligent monitoring based on computer vision can allow one to provide risk detection and support services that increase peoples autonomy at home. In the present work, a level-based visualisation scheme is proposed to provide visual privacy when human intervention is necessary, such as at telerehabilitation and safety assessment applications. Visualisation levels are dynamically selected based on the previously modelled context. In this way, different levels of protection can be provided, maintaining the necessary intelligibility required for the applications. Furthermore, a case study of a living room, where a top-view camera is installed, is presented. Finally, the performed survey-based evaluation indicates the degree of protection provided by the different visualisation models, as well as the personal privacy preferences and valuations of the users.

Optimal joint selection for skeletal data from RGB-D devices using a genetic algorithm

The growth in interest in RGB-D devices (e.g. Microsoft Kinect or ASUS Xtion Pro) is based on their low price, as well as the wide range of possible applications. These devices can provide skeletal data consisting of 3D position, as well as orientation data, which can be further used for pose or action recognition. Data for 15 or 20 joints can be retrieved, depending on the libraries used. Recently, many datasets have been made available which allow the comparison of different action recognition approaches for diverse applications (e.g. gaming, Ambient-Assisted Living, etc.). In this work, a genetic algorithm is used to determine the contribution of each of the skeletons joints to the accuracy of an action recognition algorithm, thus using or ignoring the data from each joint depending on its relevance. The proposed method has been validated using a k-means-based action recognition approach and using the MSR-Action3D dataset for test. Results show the presented algorithm is able to improve the recognition rates while reducing the feature size.

Abnormal gait detection with RGB-D devices using joint motion history features

Human gait has become of special interest to health professionals and researchers in recent years, not only due to its relation to a persons quality of life and personal autonomy, but also due to the involved cognitive process, since deviation from normal gait patterns can also be associated to neurological diseases. Vision-based abnormal gait detection can provide support to current human gait analysis procedures providing quantitative and objective metrics that can assist the evaluation of the geriatrician, while at the same time providing technical advantages, such as low intrusiveness and simplified setups. Furthermore, recent advances in RGB-D devices allow to provide low-cost solutions for 3D human body motion analysis. In this sense, this work presents a method for abnormal gait detection relying on skeletal pose representation based on depth data. A novel spatio-temporal feature is presented that provides a representation of a set of consecutive skeletons based on the 3D location of the skeletal joints and the motions age. The corresponding feature sequences are learned using a machine learning method, namely BagOfKeyPoses. Experimentation with different datasets and evaluation methods shows that reliable detection of abnormal gait is obtained and, at the same time, an outstandingly high temporal performance is provided.

Visual Privacy by Context: A Level-Based Visualisation Scheme

In a near future, a greater number of individuals in long-term care will live alone. New solutions are needed in order to provide them support and increase their autonomy at home. Intelligent monitoring systems based on computer vision may provide a solution. However, privacy related issues must be solved beforehand. In this paper, we propose a level-based visualisation scheme to give users control about their privacy in those cases in which another person is watching the video. These visualisation levels are dynamically selected according to the context by displaying modified images in which sensitive areas are protected.

The "Good" Brother: Monitoring People Activity in Private Spaces

Population over 50 will rise by 35% until 2050. Thus, attention to the needs of the elderly and disabled is today in all developed countries one of the great challenges of social and economic policies. There is a worldwide interest in systems for the analysis of people’s activities, especially those most in need.

A Vision System for Intelligent Monitoring of Activities of Daily Living at Home

Social progress and demographic changes favor increased life expectancy and the number of people in situations of dependency. As a consequence, the demand for support systems for personal autonomy is increasing. This article outlines the vision @ home project, whose goal is the development of vision-based services for monitoring and recognition of the activity carried out by individuals in the home. Incorporating vision devices in private settings is justified by its power to extract large amounts of data with low cost but must safeguard the privacy of individuals. The vision system we have designed incorporates a knowledge base containing information from the environment, parameters of different cameras used, human behavior modeling and recognition, and information about people and objects. By analyzing the scene, we infer its context, and provide a privacy filter which is able to return textual information, as well as synthetic and real images.