Silvia Pfeiffer

TV anytime as an application scenario for MPEG-7

The ISO/MPEG group has identified a wide range of application scenarios [1] for their emerging MPEG-7 standard on audio-visual metadata. TV Anytime with their vision of future digital TV services [2] encompasses a large number of them. As TV Anytime has also identified metadata as one of the key requirements to realize their vision, MPEG-7 is the natural candidate to fill that role. Here, we describe technically how metadata for the TV Anytime scenario can be created using MPEG-7.

Scene Determination Based on Video and Audio Features

Determining automatically what constitutes a scene in a video is a challenging task, particularly since there is no precise definition of the term “scene”. It is left to the individual to set attributes shared by consecutive shots which group them into scenes. Certain basic attributes such as dialogs, settings and continuing sounds are consistent indicators. We have therefore developed a scheme for identifying scenes which clusters shots according to detected dialogs, settings and similar audio. Results from experiments show automatic identification of these types of scenes to be reliable.

Video blogging: content to the max

The lure of video blogging combines the ubiquitous, grassroots, Web-based journaling of blogging with the richness of expression available in multimedia. Some claim that video blogging is an important force in a future world of video journalism and a powerful technical adjunct to our existing televised news sources. Others point to the huge demands it imposes on networking resources, the lack of hard standards, and the poor usability of current video blogging systems as indicators that its doomed to fail. Like any nascent technology, video blogging has many unsolved problems. The field, however, is vibrant, the goals are fairly clear, and the challenges they pose to multimedia researchers are exciting indeed. Developing the standards and technologies for video blogging requires a combination of approaches from various areas including media representation, information retrieval, multimedia content analysis, and video summarization. Like the development of the Web and text blogging before, video blogging only come about through open development and collaboration between engineers and researchers from diverse fields. Most strikingly, it is fueled by the passion and enthusiasm of those creating content - those who go to the trouble of recording their lives and opinions within the fledgling medium, shaping it as a lively and useful resource for generations of Internet users to come.

Pause concepts for audio segmentation at different semantic levels

This paper presents work on the determination of temporal audio segmentations at different semantic levels. The segmentation algorithm draws upon the calculation of relative silences or pauses. A perceptual loudness measure is the only feature employed. An adaptive threshold is used for classification into pause and non-pause. The segmentation algorithm that determines perceptually relevant pause intervals for different semantic levels incorporates a minimum duration and a maximum interruption constraint. The influence of the different parameters on the segmentation is examined in experiments and presented in this paper. A new approach for evaluating segmentation accuracies is required. It is shown that the simple perceptual pause concept has a very high relevance when segmenting audio at different semantic levels.

Accessibility for the HTML5 element

In this paper, we describe existing implementations for putting subtitles and captions alongside the HTML5 <video> tag inside Web pages and a proposal for standardizing such approaches, which will make them interoperable and easier to be processed by automated tools. Since video and audio are fundamental data types that any Web user will want to make use of nowadays -- if young or old -- if impaired or not -- a standard means of providing accessibility to such fundamental data types is of utmost importance as part of the standardization process of the <video> and <audio> tags. The proposal is an outcome of a Mozilla grant and of extensive discussions within the Xiph accessibility group. Ideas discussed on the WHATWG mailing list have also been taken into account. Standardization work for this is ongoing.

A Survey of MPEG-1 Audio, Video and Semantic Analysis Techniques

Digital audio & video data have become an integral part of multimedia information systems. To reduce storage and bandwidth requirements, they are commonly stored in a compressed format, such as MPEG-1. Increasing amounts of MPEG encoded audio and video documents are available online and in proprietary collections. In order to effectively utilise them, we need tools and techniques to automatically analyse, segment, and classify MPEG video content. Several techniques have been developed both in the audio and visual domain to analyse videos. This paper presents a survey of audio and visual analysis techniques on MPEG-1 encoded media that are useful in supporting a variety of video applications. Although audio and visual feature analyses have been carried out extensively, they become useful to applications only when they convey a semantic meaning of the video content. Therefore, we also present a survey of works that provide semantic analysis on MPEG-1 encoded videos.

Implementing the media fragments URI specification

In this paper, we describe two examples of implementations of the Media Fragments URI specification which is currently being developed by the W3C Media Fragments Working Group. The groups mission is to create standard addressing schemes for media fragments on the Web using Uniform Resource Identifiers (URIs). We describe two scenarios to illustrate the implementations. More specifically, we show how User Agents (UA) will either be able to resolve media fragment URIs without help from the server, or will make use of a media fragments-aware server. Finally, we present some ongoing discussions and issues regarding the implementation of the Media Fragments specification.

A URI-based approach for addressing fragments of media resources on the Web

To make media resources a prime citizen on the Web, we have to go beyond simply replicating digital media files. The Web is based on hyperlinks between Web resources, and that includes hyperlinking out of resources (e.g., from a word or an image within a Web page) as well as hyperlinking into resources (e.g., fragment URIs into Web pages). To turn video and audio into hypervideo and hyperaudio, we need to enable hyperlinking into and out of them. The W3C Media Fragments Working Group is taking on the challenge to further embrace W3C’s mission to lead the World Wide Web to its full potential by developing a Media Fragment protocol and guidelines that ensure the long-term growth of the Web. The major contribution of this paper is the introduction of Media Fragments as a media-format independent, standard means of addressing media resources using URIs. Moreover, we explain how the HTTP protocol can be used and extended to serve Media Fragments and what the impact is for current Web-enabled media formats.

The Continuous Media Web: a distributed multimedia information retrieval architecture extending the World Wide Web

The World Wide Web, with its paradigms of surfing and searching for information, has become the predominant system for computer-based information retrieval. Media resources, however information-rich, only play a minor role in providing information to Web users. While bandwidth (or the lack thereof) may be an excuse for this situation, the lack of surfing and searching capabilities on media resources are the real issue. We present an architecture that extends the Web to media, enabling existing Web infrastructures to provide seamless search and hyperlink capabilities for time-continuous Web resources, with only minor extensions. This makes the Web a true distributed information system for multimedia data. The article provides an overview of the specifications that have been developed and submitted to the IETF for standardization. It also presents experimental results with prototype applications.

Survey of compressed domain audio features and their expressiveness

We give an overview of existing audio analysis approaches in the compressed domain and incorporate them into a coherent formal structure. After examining the kinds of information accessible in an MPEG-1 compressed audio stream, we describe a coherent approach to determine features from them and report on a number of applications they enable. Most of them aim at creating an index to the audio stream by segmenting the stream into temporally coherent regions, which may be classified into pre-specified types of sounds such as music, speech, speakers, animal sounds, sound effects, or silence. Other applications centre around sound recognition such as gender, beat or speech recognition.