Visual Semantic Image Recommendation

Abstract

Image recommendation is an essential component of the modern online image sharing applications (e.g., Flickr), aiming to provide users with interesting images for further exploration. However, most existing approaches tend to treat the image in question as a single object, ignoring the important semantics of the sub-objects within the image. The loss of these semantic objects may lead to the misunderstanding of the user preference toward an image. In this paper, we propose a novel pairwise preference model, called Visual Semantic Model (VSM), to address this issue for a better recommendation. Specifically, we model the image representation by combining the feature embeddings of the fine-grained image objects, the weights of which may be distinct for different users. Then, we enhance the user modeling by taking into account the interacted images along with their relative importance. Two attention networks on both object and image levels are adapted to compute the weights of objects and images, respectively. The experimental results on the Flickr dataset show that our VSM model achieves significant improvements (around 9.18% on average in terms of Precision@5) over the state-of-the-art approaches in terms of the recommendation accuracy.