A Cross-Modal Image-Text Retrieval System with Deep Learning

Abstract

With the generation of massive data, cross-modal image and text retrieval has attracted more and more attention. However, there is a heterogeneity gap between the data of different modalities, which cannot be measured directly. To solve this problem, the researchers project the data of different modalities into the common representation space to compensate for the differences between the data of different modalities. These studies focus on improving the mean average retrieval accuracy. Few research methods have been directly applied to the design of cross-modal retrieval systems. Therefore, this paper proposes a cross-modal image-text retrieval system based on deep learning. First, we design a cross-modal image and text retrieval system to realize the application of the retrieval system. Then, we apply deep learning to the cross-modal retrieval system to make full use of the research results of cross-modal retrieval. Finally, the designed system can carry out the bidirectional retrieval of images and texts to meet the requirements of images and texts as queries. The results show that the proposed cross-modal image and text retrieval system is capable of bidirectional retrieval of images and text, which is helpful to use different modalities images and text data.