Behavior Recognition Algorithm Based on the Fusion of SE-R3D and LSTM Network

Abstract

In view of the fact that the existing behavior recognition algorithms cannot fully extract abstract behavior features, this paper proposes a SE-R3D-LSTM behavior recognition algorithm based on 3D residual convolutional neural network (R3D), which integrates Squeeze-and-excitation network (SENet)and long short-term memory (LSTM). First of all, a residual module is added to the 3D Convolutional Neural Network (3D-CNN) to avoid problems such as gradient dispersion caused by the deepening of the network layer; Secondly, not only the global average pooling layer but also the global maximum pooling layer is used in the SENet network, which can fully extract global information and achieve feature calibration. In the meantime, expand the SENet network to three-dimensional, which can make the connection of the spatiotemporal feature channels closer. Afterwards, the 3D-SE module is introduced into the R3D network, which can enhance the effective spatiotemporal features and suppress the invalid spatiotemporal features; Since, because LSTM can perform timing modeling on high-level features and learn more effective feature information, the LSTM network is introduced into the SE-R3D network. Finally, Softmax is used for classification. Experimental results show that the recognition rate of the SE-R3D-LSTM network on the UCF101 data set reaches 96.5%.