Object-Orient Semantic-to-Visual Generation Model for Stickers Synthetizing

Abstract

Stickers are used by social users to express online chat. The retrieval, matching, and generation of stickers have significant research and application value. However, the current stickers are mainly obtained through “semantic-visual” cross-modal search matching in a pre-defined sticker library. The retrieved images from the pre-defined sticker library are fixed and limited, and often do not conform to the contextual semantics of chatting. This work focuses on synthesizing non-existent stickers instead of retrieving images. The Object-Orient Semantic-to-Visual Generation Model(OSVGAN) is proposed to instantly synthesize the corresponding sticker according to the user s semantic description. First, the intrinsic semantic relationship of each object in the sticker is established by an Object-Orient Meta Model. Then a Semantic-to-Visual model is used to synthesize corresponding images according to each object s text feature, and a Fusion Model fuses each object image to form a complete image. Finally, to enhance the semantic consistency of the generated images, the text-image alignment module(T2I) is applied to the Semantic-to-Visual model and the Fusion Model. OSVGAN can effectively reduce the demand for training samples. On the sticker dataset, our method has achieved good semantic consistency, and is comparable to the existing methods in terms of synthetic image quality.