Jacques Boonaert

Human-object interaction recognition by learning the distances between the object and the skeleton joints

In this paper we present a fully automatic approach for human-object interaction recognition from depth sensors. Towards that goal, we extract relevant frame-level features such as inter-joint distances and joint-object distances that are suitable for real time action recognition. These features are insensitive to position and pose variation. Experiments conducted on ORGBD dataset following state-of-the-art settings show the effectiveness of the proposed approach.

3D real-time human action recognition using a spline interpolation approach

This paper presents a novel descriptor based on skeleton information provided by RGB-D videos for human action recognition. These features are obtained, considering the motion as continuous trajectories of skeleton joints. With the discrete information of skeleton joints position, a cubic-spline interpolation is applied to joints position, velocity and acceleration components. The training and classification steps are done using a linear SVM. In the literature, many human motion descriptors based on RGB-D cameras had already been proposed with good accuracy, but with a high computational time. The main interest of this proposed approach is its ability to calculate human motion descriptors with a low computation cost while such a descriptor leads to an acceptable accuracy of recognition. Thus, this approach can be adapted to human computer interaction applications. For the purpose of validation, we apply our method to the challenging benchmark MSR-Action3D and introduce a new indicator which is the ratio between accuracy and execution time per descriptor. Using this criterion, we show that our algorithm outperforms the state-of-art methods in terms of the combined information of rapidity and accuracy.

Human action classification using surf based spatio-temporal correlated descriptors

This paper proposes a method for human action classification by utilizing correlations between SURF based descriptors. This approach provides us a novel type of descriptor that can be used for action classification. The method proposed is tested using an SVM classification technique. For evaluation purposes, the KTH action recognition dataset, which is a standard benchmark for this area is used as it is one of the most well known and challenging dataset. The method proposed was able to successfully classify different action classes.

User in the Loop: Adaptive Smart Homes Exploiting User Feedback—State of the Art and Future Directions

Due to the decrease of sensor and actuator prices and their ease of installation, smart homes and smart environments are more and more exploited in automation and health applications. In these applications, activity recognition has an important place. This article presents a general architecture that is responsible for adapting automation for the different users of the smart home while recognizing their activities. For that, semi-supervised learning algorithms and Markov-based models are used to determine the preferences of the user considering a combination of: (1) observations of the data that have been acquired since the start of the experiment and (2) feedback of the users on decisions that have been taken by the automation. We present preliminarily simulated experimental results regarding the determination of preferences for a user.

Human Object Interaction Recognition Using Rate-Invariant Shape Analysis of Inter Joint Distances Trajectories

Human action recognition has emerged as one of the most challenging and active areas of research in the computer vision domain. In addition to pose variation and scale variability, high complexity of human motions and the variability of object interactions represent additional significant challenges. In this paper, we present an approach for human-object interaction modeling and classification. Towards that goal, we adopt relevant frame-level features, the inter-joint distances and joints-object distances. These proposed features are efficiently insensitive to position and pose variation. The evolution of the these distances in time is modeled by trajectories in a high dimension space and a shape analysis framework is used to model and compare the trajectories corresponding to human-object interaction in a Riemannian manifold. The experiments conducted following state-of-the-art settings and results demonstrate the strength of the proposed method. Using only the skeletal information, we achieve state-of-the-art classification results on the benchmark dataset.

Detection of Abnormal Gait from Skeleton Data

Human gait analysis has becomes of special interest to computer vision community in recent years. The recently developed commodity depth sensors bring new opportunities in this domain.In this paper, we study the human gait using non intrusive sensors (Kinect 2) in order to classify normal human gait and abnormal ones. We propose the evolution of inter-joints distances as spatio temporal intrinsic feature that have the advantage to be robust to location. We achieve 98% success to classify normal and abnormal gaits and show some relevant features that are able to distinguish them.

Distances evolution analysis for online and off-line human object interaction recognition

Abstract Human action recognition in 3D sequences is one of the most challenging and active areas of research in the computer vision domain. However designing automatic systems that are robust to significant variability due to object combinations and high complexity of human motions is more challenging in addition to the typical requirements such as rotation, translation, and scale invariance is challenging task. In this paper, we propose a spatio-temporal modeling of human-object interaction videos for online and off-line recognition. The inter-joint distances and the object are considered as low-level features for online classification. For off-line recognition, we propose rate-invariant classification of full video and early recognition. A shape analysis of trajectories of the inter-joint and object-joints distances is proposed for this end. The experiments conducted following state-of-the-art settings using MSR Daily Activity 3D Dataset and Online RGBD Action Dataset and on a new multi-view dataset for human object interaction demonstrate that the proposed approach is effective and discriminative for human object interaction classification as demonstrated here.

An extension of kernel learning methods using a modified Log-Euclidean distance for fast and accurate skeleton-based Human Action Recognition

Abstract In this article, we introduce a fast, accurate and invariant method for RGB-D based human action recognition using a Hierarchical Kinematic Covariance (HKC) descriptor. Recently, non singular covariance matrices of pattern features which are elements of the space of Symmetric Definite Positive (SPD) matrices, have been proven to be very efficient descriptors in the field of pattern recognition. However, in the case of action recognition, singular covariance matrices cannot be avoided because the dimension of features could be higher than the number of samples. Such covariance matrices (non singular and singular) belong to the space of Symmetric Positive semi-Definite (SPsD) matrices. Thus, in order to classify actions, we propose to adapt kernel methods such as Support Vector Machines (SVM) and Multiple Kernel Learning (MKL) to the space of SPsD matrices by using a perturbed Log-Euclidean distance (Arsigny et al., 2006). The mathematical validity of this perturbed distance (called Modified Log-Euclidean distance) for SPsD is therefore studied. The offline experiments are conducted on three challenging benchmarks, namely MSRAction3D, UTKinect and Multiview3D datasets. A fair comparison demonstrates that our approach competes with state-of-the-art methods in terms of accuracy and computational latency. Finally, our method is extended to an online scenario and experiments on MSRC12 prove the efficiency of this extension.

Application of MID-SVM for online person identification using appearance-based features

Person identification is an important but still challenging problem in video surveillance. This work designs a complete automatic appearance-based person identification system in online setting. The proposed system consists of three modules: background extraction and silhouette extraction; feature extraction and selection; and person identification. Grey-world normalized color features and Haralick texture features are extracted as initial feature subset. The multi-category incremental and decremental SVM (MID-SVM) algorithm is used to adaptively classify persons with the advantage of training only with few initial images and updating the needed parts. A new video database with 22 persons is created in real-life environments. The experimental results show that the proposed system succeed in person identification. In order to improve the performance of this system, a comparison of RGB and HSV colors is performed and the comparison experimental results indicate that the initial feature with HSV color and Haralick texture features works better.Person identification is an important but still challenging problem in video surveillance. This work designs a completely automatic appearance-based person identification system, which has the ability to achieve new person discovery and classification. The proposed system consists of three modules: background and silhouette separation; feature extraction and selection; and online person identification. The Self-Adaptive Kernel Machine (SAKM) algorithm is used to differentiate existing persons who can be classified from new persons who have to be learnt and added. A new video database with 22 persons is created in real-life environments. The experimental results show that the proposed system achieves satisfying recognition rates of over 90% on person classification with novelty identification.

Online person identification and new person discovery using appearance features

Person identification is an important but still challenging problem in video surveillance. This work designs a complete automatic appearance-based person identification system in online setting. The proposed system consists of three modules: background extraction and silhouette extraction; feature extraction and selection; and person identification. Grey-world normalized color features and Haralick texture features are extracted as initial feature subset. The multi-category incremental and decremental SVM (MID-SVM) algorithm is used to adaptively classify persons with the advantage of training only with few initial images and updating the needed parts. A new video database with 22 persons is created in real-life environments. The experimental results show that the proposed system succeed in person identification. In order to improve the performance of this system, a comparison of RGB and HSV colors is performed and the comparison experimental results indicate that the initial feature with HSV color and Haralick texture features works better.Person identification is an important but still challenging problem in video surveillance. This work designs a completely automatic appearance-based person identification system, which has the ability to achieve new person discovery and classification. The proposed system consists of three modules: background and silhouette separation; feature extraction and selection; and online person identification. The Self-Adaptive Kernel Machine (SAKM) algorithm is used to differentiate existing persons who can be classified from new persons who have to be learnt and added. A new video database with 22 persons is created in real-life environments. The experimental results show that the proposed system achieves satisfying recognition rates of over 90% on person classification with novelty identification.