Low-Rank Nonlocal Representation for Remote Sensing Scene Classification

Abstract

The nonlocal mechanism (NM) has shown its effectiveness in many real-world applications. However, it is usually criticized for its costly computation complexity in time and space, which are both O(H²W²), where Hx W is the spatial dimension of input feature maps in height and width. In this letter, we first show that the NM can be expressed as a low-rank representation by theory. Then we propose a down-to-earth low-complexity global context acquisition mechanism, termed as the low-rank nonlocal representation (LNR), whose complexity in time and space are both approximatively O(HW). LNR is a general module that can be deployed on an arbitrary convolutional neural network (CNN) hierarchy for any visual recognition tasks. To demonstrate its superiority, experiments are carried out on four standard remote sensing scene classification benchmarks. Experimental results show that our proposed LNR can significantly reduce the computation cost by boosting the performance gain. Implementing LNR on the classical ResNet, in particular, can reduce at most 3.93 M model parameters.