Isao Otsuka

An SVM Framework for Genre-Independent Scene Change Detection

We present a novel genre-independent SVM framework for detecting scene changes in broadcast video. Our framework works on content from a diverse range of genres by allowing sets of features, extracted from both audio and video streams, to be combined and compared automatically without the use of explicit thresholds. For ground truth, we use hand-labeled video scene boundaries from a wide variety of broadcast genres to generate positive and negative samples for the SVM. Our experiments include high-and low-level audio features such as semantic histograms and distances between Gaussian models, as well as video features such as shot cut positions. We evaluate the importance of these measures in a structured framework, with performance comparisons obtained via ROC curves. We achieve over 70% detection rate for 10% false positive rate on our corpus of over 7.5 hours of data collected from news, talk shows, sitcoms, dramas, music videos, and how-to shows.

Broadcast Video Program Summarization using Face Tracks

We present a novel video summarization and skimming technique using face detection on broadcast video programs. We take the faces in video as our primary target as they constitute the focus of most consumer video programs. We detect face tracks in video and define face-scene fragments based on start and end of face tracks. We define a fast-forward skimming method using frames selected from fragments, thus covering all the faces and their interactions in the video program. We also define novel constraints for a smooth and visually representative summary, and construct longer but smoother summaries

Audio-Assisted video browsing for DVD recorders

We present an audio-assisted video browsing system for a Hard Disk Drive (HDD) enhanced DVD recorder. We focus on our sports highlights extraction based on audio classification. We have systematically established that sports highlights are indicated by the presence of audience reaction such as cheering, applause and the commentators excited speech. That enables us to develop a common highlights extraction technique, based on detection of audience reaction, for five different sports, viz. soccer, baseball, golf, sumo wrestling and horseracing. Our extraction accuracy is high. Furthermore, the percentage duration of the audience reaction gives us a simple importance measure for each of the highlights. We can then get a summary of any desired length by appropriately choosing a threshold for the importance measure. We process the AC-3 audio directly thereby enabling simple integration of our technique into our target platform.

Sports Program Boundary Detection

In the recent years, consumer devices that can record broadcast video have become prevalent. Such devices rely on the program guide information about a program start time and end time for recording. However, sports broadcasts can run over the specified time sometimes. In this paper, we propose a framework based on an audio classification framework to correctly detect the end of a sport broadcast so as to enable complete recording of the game. Our experimental results show that the proposed algorithm can detect sports program boundaries with a high accuracy