A Human Activity Recognition Approach Based on Skeleton Extraction and Image Reconstruction

Abstract

Human Activity Recognition (HAR) has wide applications in surveillance, assisted living and robotic systems that interact with people. Traditional methods for HAR usually input source images directly. Various factors of these direct source images can affect the accuracy of recognition, such as heights, weights, poses, angles and whether being obscured or not. Most of the latest research on HAR has mainly focused on the use of 2D skeleton data, extracted from videos captured by standard cameras. Inspired by this trend, we propose a novel way of using skeleton data in this paper. As a pre-processing phase, we adopt the OpenPose library to extract 2D positions of human skeleton joints, then reconstruct, or synthesize, the skeleton image from these joints, together with coloring to encode the categories of different human parts. The reconstructed images are put into a Convolutional Neural Network (CNN) structure for classification, to obtain the recognition result. We implement the typical four activity classes, i.e., squat, stand, walk and work, in our experiments. Compared with feeding the original complex source images, our approach reduces the complexity of the neural network thus the time consumption, and improves the recognition accuracy significantly. All the images that we used for training and testing are collected from the real public places. We achieved a recognition accuracy of 97.3% in our experiments, indicating that our method has very good performance. Thanks to the lightning speed of the OpenPose library, our method can be carried out in real-time. The demonstration of using synthesized input to bridge existing technologies also has its general significance.