Thomas Sikora

Overview of the MPEG-7 standard

MPEG-7, formally known as the Multimedia Content Description Interface, includes standardized tools (descriptors, description schemes, and language) enabling structural, detailed descriptions of audio-visual information at different granularity levels (region, image, video segment, collection) and in different areas (content description, management, organization, navigation, and user interaction). It aims to support and facilitate a wide range of applications, such as media portals, content broadcasting, and ubiquitous multimedia. We present a high-level overview of the MPEG-7 standard. We first discuss the scope, basic terminology, and potential applications. Next, we discuss the constituent components. Then, we compare the relationship with other standards to highlight its capabilities.

The MPEG-7 visual standard for content description-an overview

The MPEG-7 visual standard under development specifies content-based descriptors that allow users or agents (or search engines) to measure similarity in images or video based on visual criteria, and can be used to efficiently identify, filter, or browse images or video based on visual content. More specifically, MPEG-7 specifies color, texture, object shape, global motion, or object motion features for this purpose. This paper outlines the aim, methodologies, and broad details of the MPEG-7 standard development for visual content description.

MPEG digital video-coding standards

The efficient digital representation of image and video signals has been the subject of considerable research over the past 20 years. Digital video-coding technology has developed into a mature field and products have been developed that are targeted for a wide range of emerging applications, such as video on demand, digital TV/HDTV broadcasting, and multimedia image/video database services. With the increased commercial interest in video communications, the need for international image- and video-compression standards arose. To meet this need, the Moving Picture Experts Group (MPEG) was formed to develop coding standards. MPEG-1 and MPEG-2 video-coding standards have attracted much attention worldwide, with an increasing number of very large scale integration (VLSI) and software implementations of these standards becoming commercially available. MPEG-4, the most recent MPEG standard that is still under development, is targeted for future content-based multimedia applications. We provide an overview of the MPEG video-coding algorithms and standards and their role in video communications. We review the basic concepts and techniques that are relevant in the context of the MPEG video-compression standards and outline MPEG-1 and MPEG-2 video-coding algorithms. The specific properties of the standards related to their applications are presented, and the basic elements of the forthcoming MPEG-4 standard are also described. We also discuss the performance of the standards and their success in the market place.

Shape-adaptive DCT for generic coding of video

A low complexity shape-adaptive DCT algorithm suitable for coding pels in arbitrarily shaped image segments is introduced. In contrast to other techniques described in literature the proposed algorithm is based on predefined orthogonal sets of DCT basis functions and does not require more computations than a normal block DCT. It is shown that the shape-adaptive DCT algorithm can be easily incorporated into existing block-based JPEG, H.261, or MPEG coding schemes. Thus segment or object based coding of images and video can be provided with backward compatibility to existing coding standards. As an important feature, with the proposed technique additional content based functionalities currently discussed in the MPEG-4 standardization phase can be readily achieved. >

Trends and Perspectives in Image and Video Coding

The objective of the paper is to provide an overview on recent trends and future perspectives in image and video coding. Here, I review the rapid development in the field during the past 40 years and outline current state-of-the art strategies for coding images and videos. These and other coding algorithms are discussed in the context of international JPEG, JPEG 2000, MPEG-1/2/4, and H.261/3/4 standards. Novel techniques targeted at achieving higher compression gains, error robustness, and network/device adaptability are described and discussed.

Long-term global motion estimation and its application for sprite coding, content description, and segmentation

We present a new technique for long-term global motion estimation of image objects. The estimated motion parameters describe the continuous and time-consistent motion over the whole sequence relatively to a fixed reference coordinate system. The proposed method is suitable for the estimation of affine motion parameters as well as for higher order motion models like the parabolic model-combining the advantages of feature matching and optical flow techniques. A hierarchical strategy is applied for the estimation, first translation, affine motion, and finally higher order motion parameters, which is robust and computationally efficient. A closed-loop prediction scheme is applied to avoid the problem of error accumulation in long-term motion estimation. The presented results indicate that the proposed technique is a very accurate and robust approach for long-term global motion estimation, which can be used for applications such as MPEG-4 sprite coding or MPEG-7 motion description. We also show that the efficiency of global motion estimation can be significantly increased if a higher order motion model is applied, and we present a new sprite coding scheme for on-line applications. We further demonstrate that the proposed estimator serves as a powerful tool for segmentation of video sequences.

Image sequence analysis for emerging interactive multimedia services-the European COST 211 framework

Flexibility and efficiency of coding, content extraction, and content-based search are key research topics in the field of interactive multimedia. Ongoing ISO MPEG-4 and MPEG-7 activities are targeting standardization to facilitate such services. European COST Telecommunications activities provide a framework for research collaboration. At present a significant effort of the COST 211/sup ter/ group activities is dedicated toward image and video sequence analysis and segmentation-an important technological aspect for the success of emerging object-based MPEG-4 and MPEG-7 multimedia applications. The current work of COST 211 is centered around the test model, called the analysis model (AM). The essential feature of the AM is its ability to fuse information from different sources to achieve a high-quality object segmentation. The current information sources are the intermediate results from frame-based (still) color segmentation, motion vector based segmentation, and change-detection-based segmentation. Motion vectors, which form the basis for the motion vector based intermediate segmentation, are estimated from consecutive frames. A recursive shortest spanning tree (RSST) algorithm is used to obtain intermediate color and motion vector based segmentation results. A rule-based region processor fuses the intermediate results; a postprocessor further refines the final segmentation output. The results of the current AM are satisfactory.

Digital video coding standards and their role in video communications

The efficient digital representation of image and video signals has been subject of considerable research over the past 20 years. With the growing availability of digital transmission links, progress in signal processing, VLSI technology and image compression research, visual communications has become more feasible than ever. Digital video coding technology has developed into a mature field and a diversity of products has been developed-targeted for a wide range of emerging applications, such as video on demand, digital TV/HDTV broadcasting, and multimedia image/video database services. With the increased commercial interest in video communications the need for international image and video coding standards arose. Standardization of video coding algorithms holds the promise of large markets for video communication equipment. Interoperability of implementations from different vendors enables the consumer to access video from a wider range of services and VLSI implementations of coding algorithms conforming to international standards can be manufactured at considerably reduced costs. The purpose of this paper is to provide an overview of todays image and video coding standards and their role in video communications. The different coding algorithms developed for each standard are reviewed and the commonalities between the standards are discussed. >

Low complexity shape-adaptive DCT for coding of arbitrarily shaped image segments

Abstract A low complexity shape-adaptive DCT transform algorithm for coding pels in arbitrarily shaped image segments is presented. The proposed algorithm is compared to the well established generalized shape-adaptive transform method introduced by Gilge et al. in terms of transform efficiency and computational complexity. Results obtained under both theoretical and experimental conditions show that the new algorithm achieves a transform efficiency close to that of the Gilge method with considerably reduced computational complexity. The proposed shape-adaptive DCT algorithm was implemented into a standard MPEG-1 coder to provide object or segment based coding of images and video with additional content-based functionality. The extended MPEG-1 object based coding scheme can handle generic input sequences and can readily provide MPEG-1 backward compatibility if no contour data is transmitted for a given video sequence. Results for INTRA coding of images indicate that the algorithm allows efficient coding over a wide range of coding parameters — thus providing means for generic coding of segmented video between very high and very low bit rates. It is further shown that some of the content-based based functionalities currently discussed in MPEG-4 can be provided efficiently using the proposed object based coding scheme.

3D Videocommunication: Algorithms, concepts and real-time systems in human centred communication

List of Contributors. Symbols. Abbreviations. Introduction by O. Schreer, P. Kauff, T. Sikora. SECTION I: APPLICATIONS OF 3D VIDEOCOMMUNICATION. 1. History of Telepresence (W.A. IJsselsteijn). 2. 3D TV broadcasting (C. Fehn). 3. 3D in Content Creation and Post-Production (O. Grau). 4. Free Viewpoint Systems (M. Tanimoto). 5. Immersive Videoconferencing (P. Kauff, O. Schreer). SECTION II: 3D DATA REPRESENTATION AND PROCESSING. 6. Fundamentals of Multiple View Geometry (E. Trucco, S. Ivekovic, A. Fusiello). 7. Stereo Analysis (N. Atzpadin, J. Mulligan). 8. Reconstruction of Volumetric 3D Models (P. Eisert). 9. View Synthesis and Rendering Methods (R. Koch, J.-F. Evers-Senne). 10. 3D Audio Capture and Analysis (M. Schwab, P. Noll). 11. Coding and Standardization (A. Smolic, Th. Sikora). SECTION III: 3D REPRODUCTION. 12. Human Factors of 3D Displays (W.A. IJsselsteijn, P.J.H. Seuntiens, L.M.J. Meesters). 13. 3D Displays (S. Pastoor). 14. Mixed Reality Displays (S. Pastoor, C. Conomis). 15. Spatialised Audio and 3D Audio Rendering (Th. Sporer, S. Brix). SECTION IV: 3D DATA SENSORS. 16. Sensor-Based Depth Capturing (Joao G.M. Goncalves, V. Sequeira). 17. Tracking and User Interface for Mixed Reality (T. Ebrahimi, Y. Abdeljaoued, D. Marimon). Index.