A dynamic CNN pruning method based on matrix similarity

Abstract

Network pruning is one of the predominant approaches for deep model compression. Pruning large neural networks while maintaining their performance is often desirable because space and time complexity are reduced. Current pruning methods mainly focus on the importance of filters in the whole task. Different from previous methods, this paper focuses on the similarity between the filters or feature maps of the same layer. Firstly, cosine similarity is used as the matrix similarity measure to measure the similarity between channels, guiding the network to prune. Secondly, the proposed method is, respectively, applied to filters and feature maps pruning, and the pruning effects in different layers are summarized. Finally, we propose a method to set the pruning rate dynamically according to the situation of each layer. Our method obtains extremely sparse networks with virtually the same accuracy as the reference networks on the CIFAR-10 and ImageNet ILSVRC-12 classification tasks. On CIFAR-10, our network achieves the 52.70% compression ratio on ResNet-56 and increases only 0.13% on top-1 error.