Chung-Yi Weng

RoleNet: Movie Analysis from the Perspective of Social Networks

With the idea of social network analysis, we propose a novel way to analyze movie videos from the perspective of social relationships rather than audiovisual features. To appropriately describe roles relationships in movies, we devise a method to quantify relations and construct roles social networks, called RoleNet. Based on RoleNet, we are able to perform semantic analysis that goes beyond conventional feature-based approaches. In this work, social relations between roles are used to be the context information of video scenes, and leading roles and the corresponding communities can be automatically determined. The results of community identification provide new alternatives in media management and browsing. Moreover, by describing video scenes with roles context, social-relation-based story segmentation method is developed to pave a new way for this widely-studied topic. Experimental results show the effectiveness of leading role determination and community identification. We also demonstrate that the social-based story segmentation approach works much better than the conventional tempo-based method. Finally, we give extensive discussions and state that the proposed ideas provide insights into context-based video analysis.

Tiling slideshow

This paper presents a new medium, called tiling slideshow, to display photos in a tile-like manner, coordinating with the pace of background music. In contrast to the conventional photo slideshow, multiple photos that have similar characteristics are well arranged and displayed at the same layout. Motivated by the concepts of technical writing, each displaying layout is composed of a larger topic photo and several small-size supportive photos. Based on this idea, the proposed tiling slideshow system consists of three major components: image clustering, music analyzer, and layout organizer. Given the limited displaying space, we consider the context and relationship between photos and model the layout organization as a constrainted optimization problem. Experiments on real consumer photograph collections show that the novel displaying method gives users more pleasant browsing experience than the methods that focus only on single photograph display.

Audiovisual slideshow: present your journey by photos

This demonstration presents a novel way to systematically display photos and enhance the viewing experience of photo browsing. In contrast to conventional photo slideshow, multiple photos that have similar characteristics are well arranged and displayed at the same layout. Moreover, the displaying pace is coordinated with the beat of the user-selected incidental music. To automatically generate the audiovisual slideshow, we develop a system that consists of three main components: photo analysis, music analysis, and audiovisual composition. Audiovisual content analysis and cross-media synchronization issues are addressed in this work. This novel demonstration is especially suitable to present photos taken in a journey. It vigorously presents the delights of traveling and helps us recall or experience the trip.

Low-bitrate video quality enhancement by frame rate up-conversion and adaptive frame encoding

Frame rate up-conversion (FRUC) is a useful technique for a lot of practical applications, such as display format conversion, low bitrate video coding and slow motion playback. Unlike traditional approaches, such as frame repetition or linear frame interpolation, motion-compensated frame interpolation (MCFI) technique which takes block motion into account is regarded as a more efficient scheme. By considering the deficiencies in previous works, new criteria and coding schemes for enhancing motion derivation and interpolation processes are suggested. We then integrate the proposed MCFI scheme into the decoding process of the latest coding standard, H.264/AVC. In addition, adaptive frame skip is fulfilled at the encoder side to maximize the power of MCFI in video coding applications. As a result, the encoder can adopt the MCFI dynamically and can decide whether the input frame should be coded or dropped then interpolated. Experimental results show that our proposal indeed enhances the overall quality, both subjectively and objectively, especially for the low bitrate video coding.