Image registration of SAR and Optical based on salient image sub-patches

Abstract

As a fundamental and critical task in multi-source image fusion, the registration of optical image and synthetic aperture radar (SAR) image can identify corresponding identical or similar structures from two heterogeneous images. Although Pseudo Siamese network has achieved notable success in matching heterogeneous images, the network prone to mismatch when there are blurred, duplicate or similar scenes appeared in the search image. To improve the performance, in this paper, the OPT-to-SAR image pair is cut into sub-patch pairs through a pre-defined sliding window. Then a two-stage image filtering mechanism is proposed to maintain candidate sub-patches with ideal texture information. After sending all the qualified sub-patch pairs into the Pseudo Siamese network, the final matching result will be passed through a RANSAC module. In this way, the interference from invalid image areas can be reduced and the model robustness can be ensured by using the statistic information of the whole image pair. A series of experiments conducted under various data scenarios proved the effectiveness of the method.