Ma Cuixia

Video sketch summarization, interaction and cognition analysis

Video, as one typical digital media, is important for message communication. For efficient video content visualization and natural interaction such as video browsing and searching, we propose a sketch-based video summarization with fluent sketch interaction in this paper. Firstly, we present the sketch representation for video semantics, which takes the advantages of abstractness and generality of sketches. The concept of semantic sketch is proposed, which supports annotating video contents with sketches. Furthermore, an optimized layout algorithm for sketch summarization is presented. Secondly, we present the interaction techniques for sketch summarization and natural sketch gesture operations. From the viewpoint of cognitive psychology, we analyze the sketch representation, as well as the effects and relations of cognitive units in sketch interaction. Finally, user studies show that the proposed sketch summarization and sketch interaction improve user efficiency in terms of acquiring the main video content and reduce users’ cognitive load.