Multi-grained Fusion for Conditional Image Retrieval

Abstract

We tackle a new task conditional image retrieval, where the query includes an image and a corresponding additional text, with the aim to search the most similar target images in the whole gallery database. The additional text describes the additional conditions for the query image. Previous methods always represent the query pairs by fusing features at the deeper layer, which neglects the fine-grained relationships between the image and the different stages of the sentence. In this paper, we propose a Multi-Grained Fusion (MGF) module to mine the multi-grained relationships in the query pairs to fuse features more effectively. To further improve the performance, we propose an unsupervised Online Groups Matching (OGM) loss to make the feature include more identity information. Extensive experiments show that our method outperforms other state-of-the-art approaches.