Human action identification by a quality-guided fusion of multi-model feature

Abstract

Abstract Human motion recognition has become an active research area in the field of computer vision due to its wide range of implementations in domains of video monitoring, virtual reality, human–machine interaction. Dealing with the problem that the RGB images cannot provide enough depth information, a multi-modal depth neural network based on joint cost function is proposed for human motion recognition. In the architecture, the features of the RGB video frames are extracted by the 3D CNN architecture while the characteristics of human motion recognition in the SSDDI graphics utilizing depth map are extracted by the LSTM. Moreover, the model utilizes joint cost function including the cross-entropy loss and the distance constraint between the feature space of training samples and their center values within each category. The experimental results on the MSR Action 3D datasets suggest that the proposed model demonstrates a higher accuracy rate than do the other competing models.