Fusion of hyperspectral and multispectral image by dual residual dense networks

Abstract

Abstract. The spatial resolution of hyperspectral image (HSI) is severely limited as a cost for higher spectral resolution. We propose a deep learning-based HSI super resolution method named dual residual dense networks (DRDNs) to overcome the resolution limitation by fusing low-resolution (LR) HSI and high-resolution (HR) multispectral image (MSI). The proposed model uses two symmetric subnets based on residual dense block to fully exploit deep features from HR-MSI and LR-HSI, respectively. Then the features are fused through a feature fusion subnet to generate the final reconstructed HR-HSI. Our DRDNs have realized end-to-end mapping from the original input of LR-HSI and HR-MSI to the desired HR-HSI directly. All the parameters are trained through a unified framework. Some experiments are also carried out to evaluate the performance of our proposed method. The results show that our proposed method gains significant improvement in simultaneously enhancing spatial resolution and preserving spectral consistency when compared to other HSI/MSI fusion methods proposed recently.