Masanori Sano

CurioView: TV recommendations related to content being viewed

We developed a new way of viewing TV, CurioView, which uses metadata and retrieval technology to satisfy viewers curiosity by recommending wide-ranging video content related to the content the viewer is currently watching. We describe a general and expandable architecture that is based on CurioViews functions. The architecture can be applied flexibly, not just to TVs, but also to PCs and mobile terminals. We also report on the fundamental testing of a prototype system using this architecture.

Metadata production framework and metadata editor

This paper proposes the Metadata Production Framework (MPF) as a common platform for generating content-based metadata. A lot of research on extracting useful information from audiovisual content has been conducted. Recently it has been necessary to integrate these research studies to get much higher semantic metadata. The aim of MPF is to provide an environment where we can easily make these types of processes, including multimodal integration. We have also released a Metadata Editor where a user can generate content-based metadata both manually and by using automatic metadata extraction modules based on MPF.

Development and field trial of a social TV system for elderly people

As the core of an innovative business model, we seek to develop an ICT (information and communications technology) platform called Senior Cloud that will activate communication among the elderly and utilize their knowledge and labor skills in the hyper-aged society of Japan, in which the percentage of people aged 65+ currently exceeds 21% of the total population. As one aspect of this effort, we studied what leads to the activation of communication among the elderly; specifically, we extended the viewing action and social graphs of the social TV system teleda, which features VOD (Video on Demand) and SNS (Social Networking Service) functions. In this paper, we describe several experiments that verify the possibility of employing social TV systems to activate the elderly.

Generating metadata from acoustic and speech data in live broadcasting

This paper describes a method to generate metadata for TV programs in real-time by utilizing acoustic and speech data in live broadcasting. Various styles of watching TV programs can be provided by using metadata related to the content of the program. The acoustic data to be processed in our case is crowd noise in a football (soccer) stadium, and the speech data is an announcers voice. The crowd noise is closely related to not only spectator emotions but also their attention and expectations. In other words, a part in which the crowd noise rises corresponds to an important event in the game. Because the crowd noise conveys no further information about what happened in the scene, the announcers voice, after speech-to-text conversion, is processed to extract further meaning. By combining these two processes of identifying and extracting, content-based segment metadata is generated automatically. This method was applied to generating metadata for six professional football games, by which its effectiveness was verified.

Video rushes summarization utilizing retake characteristics

This paper describes the details of NHKs approach to the rushes summarization task in TRECVID 2008. From a broadcasters point of view, removing redundancy and attaining a pleasant tempo/rhythm, as well as the recall ratio, are important. Here, we introduce a novel means of representative image selection for comparison which can be applied to remove redundancy. Specifically, we make best use of normal behavior of shooting i.e., being motionless until start trigger, and after the trigger performer and camera move almost the same way (same motion.) We have built and tested a summarization system based on this technique. Although the recall ratio in our evaluation was only 33%, the reduction of duplication and the quality of tempo/rhythm were good. Improvements that can be made to our system are discussed based on the results of this evaluation.

A Social TV System for the Senior Community: Stimulating Elderly Communication Using Information and Communications Technology

The purpose of this study is to stimulate communication among the elderly population of Japan (where more than 21% of the population is aged 65 years and above) and to find ways to use their knowledge and labor skills with the aid of information and communications technology. Toward this end, we launched a joint research project called Senior Cloud in which we developed a prototype of a social TV system for the elderly. The first phase of the research project included a three-month field trial using the system in a local senior community. In this paper, we analyze the system log (posting and operational data) and the responses to questionnaires administered to obtain user impressions. These data reveal certain characteristics of communication among the elderly and methods that can stimulate it.

Image-Based Quizzes from News Video Archives

This paper proposes a method for generating image-based quizzes from news video achieves. Although there are many types of quizzes, in this work we focus on matching quizzes in which an image is to be matched to one of several choices that are statements. The key to making a successful quiz of this type is to extract choice statements that are similar but nonidentical to the true statement regarding the image. For image selection, we need to select images that are suitable for image-based quizzes. In this paper, we report some preliminary work and highlight some interesting issues in making quizzes. We also propose a method for automatically generating quizzes using feature-based clustering analyses and assess the outcomes of trials actually using our method.

Automatic Real-Time Selection and Annotation of Highlight Scenes in Televised Soccer

We describe an online method for selecting and annotating highlight scenes in soccer matches being televised. The stadium crowd noise and the play-by-play announcers voice are used as input signals. Candidate scenes for highlights are extracted from the crowd noise by dynamic thresholding and spectral envelope analysis. Using a dynamic threshold solves the problem in conventional methods of how to determine an appropriate threshold. Semantic-meaning information about the kind of play and the related team and player is extracted from the announcers commentary by using domain-based rules. The information extracted from the two types of audio input is integrated to generate segment-metadata of highlight scenes. Application of the method to six professional soccer games has confirmed its effectiveness.

The MPEG-7 Audiovisual Description Profile (AVDP) and its application to multi-view video

This paper describes a new MPEG-7 profile called AVDP (Audiovisual Description Profile). Firstly, some problems with conventional MPEG-7 profiles are described and the motivation behind the development of AVDP is explained based on requirements from broadcasters and other actors from the media industry. Secondly, the scope and functionalities of AVDP are described. Differences from the existing profiles and the basic AVDP structure and components are explained. Some useful software tools handling AVDP, including for validation and visualization are discussed. Finally the use of AVDP to represent multi-view and panoramic video content is described.

Scene-Text-Detection Method Robust Against Orientation and Discontiguous Components of Characters

Scene-text detection in natural-scene images is an important technique because scene texts contain location information such as names of places and buildings, but many difficulties still remain regarding practical use. In this paper, we tackle two problems of scene-text detection. The first is the discontiguous component problem in specific languages that contain characters consisting of discontiguous components. The second is the multi-orientation problem in all languages. To solve these two problems, we propose a connected-component-based scene-text-detection method. Our proposed method involves our novel neighbor-character search method using a synthesizable descriptor for the discontiguous-component problems and our novel region descriptor called the rotated bounding box descriptors (RBBs) for rotated characters. We also evaluated our proposed scene-text-detection method by using the well-known MSRA-TD500 dataset that includes rotated characters with discontiguous components.