Temporal Attention Networks for Multitemporal Multisensor Crop Classification

Abstract

Crop classification based on multitemporal multisensor remote sensing imagery has a great significance. As more and more Earth observation satellites are launched, it becomes easier to obtain increasingly dense multitemporal data, and the category patterns hidden in multitemporal data can be more and more finely mined and expressed. However, the traditional classification methods treat the features of different temporal periods consistently, so the classification performances are not well enough, especially for the categories with subtle phenological differences. In this paper, we propose a temporal-attention CNN-GRU (Convolutional Neural Networks and Gated Recurrent Unit Networks) approach to distinguish subtle crop differences, and the temporal attention mechanism introduced in the model can achieve the effect of enhancing phenological differences and suppressing phenological similarity. Firstly, we use the GRU networks to model the temporal correlation of the multitemporal data. And then, the temporal attention layer utilize a query module to retrieve “what is the important information” over the whole temporal sequence, so we can obtain the attention weights for each temporal period. In the experiments, multitemporal samples are collected from Sentinel-2A/B and Landsat-8. Due to the different spatial resolutions of multiband images, transposed convolution which can extract the raw spatial structure information is used for multiband features fusion. Experimental results on multitemporal data show that the proposed approach achieve the best performance compared with conventional methods, especially for the categories with similar phenological laws.