A Deep Learning-Based Heterogeneous Spatio-Temporal-Spectral Fusion: SAR and Optical Images

Abstract

Image fusion is a powerful means to integrate complementary spatio-temporal-spectral information among multi-source remote sensing images. The existing remote sensing image fusion is mostly limited to the fusion between optical images, and most of them are limited to the fusion between two sensors. Based on this, this paper proposes a heterogeneous spatio-temporal-spectral fusion method based on deep learning. Specifically, it combines the low-spatial-resolution (LR) cloudy image with the high-spatial-resolution (HR) SAR images and the HR cloud-free optical image to remove the clouds and improve the spatial resolution of the LR cloudy image. The SAR image is acquired at the same date as the LR cloudy image, while the HR cloud-free image is acquired at another date. Experiments are performed on the images of Landsat 8, Sentinel-1, and Sentinel-2. The experimental results show that the proposed method can effectively achieve the joint goal of spatial resolution improvement and cloud removal of the Landsat image.