Robert Louis Schmidt

Improvements in DCT-based video coding

We report on recent advances in traditional DCT based video coding at low bitrates. These improvements allow either an increase in coding efficiency or an increase in other functionalities. Our investigation is conducted within the framework of the ongoing work towards the MPEG-4 video standard. The ISO moving picture experts group (MPEG) is currently developing this standard after having completed the MPEG-1 and the MPEG-2 standards. The MPEG-4 video standard is addressing a number of content based as well as traditional functionalities. The development process consists of iterative refinement of the verification model via a set of well defined core experiments. Our first experiment is on improved coding efficiency of intra and uses DC and AC predictions and optimized scanning of DCT coefficients followed by a separate optimized variable length code table. Our second experiment is the study of bidirectional coding to allow additional functionality such as temporal scalability at low bit-rates. We present results of these experiments and summarize our findings.

Performance evaluation of the MPEG-4 visual coding standard

We first present an overview of the MPEG-4 video standard and its relationship to other existing as well as evolving video standards. MPEG-4 video, while introducing a new paradigm of treating each object in a scene independently, utilizes the traditional motion compensated DCT framework for coding of each object. Thus, while introducing new object based coding functionality, it is also capable of providing traditional frame-based coding. Furthermore, it supports advanced functionalities such as efficient coding of background as a sprite, robustness to channel errors, spatial and temporal scalability of arbitrary shape objects etc. Next, we evaluate the statistical performance of the MPEG-4 video under a number of selected conditions and compare it, depending on the application, with the H.263, the MPEG-1 and the MPEG-2 standards. For each traditional application, based on our limited set of experiments, MPEG-4 video appears to provide equal or better performance when compared to the most suitable existing standard addressing that application area. For the new object based applications, although MPEG-4 video when coding arbitrary shaped objects, incurs additional coding costs, perhaps with further optimization, the increased cost may be offset by improved tradeoffs in coding quality control, channel bandwidth and decoding resource adaptations.

Scene description, composition, and playback systems for MPEG-4

We discuss a few selected aspects of MPEG-4 Systems which includes a representation of description of multimedia scenes, delivery of coded multimedia objects and the scene description, composition, and synchronized playback. We first present an overview of BIFS, the MPEG-4 scene description language, as well as a brief background of VRML, on which BIFS is based. We then discuss the basics of a (nonadaptive) MPEG-4 playback system, including issues in BIFS and media decoding. The state of the software implementation in MPEG-4 of the 2D scene player and the 3D scene player is then discussed. Thus, the novel aspects of MPEG-4 Systems version 1 work which is near completion, is reviewed. We also discuss related aspects of the ongoing work on version 2 of MPEG-4 Systems. This includes additional work on scene description, referred to as advanced BIFS, and work on JavaTM based extension of MPEG- 4 System to an adaptive system referred to as MPEG-J. The work on advanced BIFS, extends the version 1 scene description by a number of additional capabilities. The work on MPEG-J involves design of architecture to control version 1 system, and set of API. Thus, the status of some of the ongoing work on MPEG-4 Systems version 2 is also reviewed.

Robustness Considerations in ISO MPEG-4 and ITU-T Mobile Video Standards

The need for mobile communications is ever increasing due to the sense of timeliness and flexibilities it offers. The increasing diversity of mobile applications is now demanding communications not only in the form of speech and data but also with synthetic and natural images, and video and is referred to as mobile multimedia. However, multimedia is expensive in the sense of its bandwidth requirement, with video being highly bandwidth intensive. Efficient compression of video is therefore critical to making any multimedia application feasible. The feasibility of mobile multimedia of acceptable quality certainly poses a significant challenge. This is so because wireless channels impose a fairly harsh environment for multimedia communications, and while the goal of compression is to squeeze redundancy out of signals to fit them on limited available bandwidth, the requirements for robust delivery necessitate some amount of redundancy. As in the case for wired or wireless environments, the success of multimedia terminals, products or services depends on many factors, of particular significance is interworking which is facilitated by standardization.

MPEG-4 playback of multimedia objects searchable by their description

In this paper, from the standpoint of their potential for multimedia applications/services we examine key new elements of the MPEG-4 standard as well as concepts leading to the ongoing work for the MPEG-7 standard. We first examine generic requirements common to several practical future multimedia applications/services. We then identify and examine key new features of the MPEG-4 standard such as, image/video objects, facial animation, advanced sound, integration with the web, multi-user worlds and Java control. Next, we briefly review the goal, requirements and progress in the ongoing work on the MPEG-7 standard. We then discuss the player we intend to use which integrates a number of key features of MPEG-4 and search and retrieval concepts similar to that of MPEG-7 standard, as well as how to create content for the player. Next, we discuss an example multimedia application that we have developed. Finally, we summarize the key highlights of the paper.

Experiments in MPEG-4 content authoring, browsing, and streaming

In this paper, within the context of the MPEG-4 standard we report on preliminary experiments in three areas -- authoring of MPEG-4 content, a player/browser for MPEG-4 content, and streaming of MPEG-4 content. MPEG-4 is a new standard for coding of audiovisual objects; the core of MPEG-4 standard is complete while amendments are in various stages of completion. MPEG-4 addresses compression of audio and visual objects, their integration by scene description, and interactivity of users with such objects. MPEG-4 scene description is based on VRML like language for 3D scenes, extended to 2D scenes, and supports integration of 2D and 3D scenes. This scene description language is called BIFS. First, we introduce the basic concepts behind BIFS and then show with an example, textual authoring of different components needed to describe an audiovisual scene in BIFS; the textual BIFS is then saved as compressed binary file/s for storage or transmission. Then, we discuss a high level design of an MPEG-4 player/browser that uses the main components from authoring such as encoded BIFS stream, media files it refers to, and multiplexed object descriptor stream to play an MPEG-4 scene. We also discuss our extensions to such a player/browser. Finally, we present our work in streaming of MPEG-4 -- the payload format, modification to client MPEG-4 player/browser, server-side infrastructure and example content used in our MPEG-4 streaming experiments.