Barry G. Haskell

Constraints on variable bit-rate video for ATM networks

Constraints on the encoded bit rate of a video signal that are imposed by a channel and encoder and decoder buffers are considered. Conditions that ensure that the video encoder and decoder buffers do not overflow or underflow when the channel can transmit a variable bit rate are presented. Using these conditions and a commonly proposed network-user contract, the effect of a (BISDN) network policing function on the allowable variability in the encoded video bit rate is examined. It is shown how these ideas might be implemented in a system that controls both the encoded and transmitted bit rates. The performance of video that has been encoded using the derived constraints for the leaky bucket channel is presented. >

Image and video coding-emerging standards and beyond

Discusses coding standards for still images and motion video. We first briefly discuss standards already in use, including: Group 3 and Group 4 for bilevel fax images; JPEG for still color images; and H.261, H.263, MPEG-1, and MPEG-2 for motion video. We then cover newly emerging standards such as JBIG1 and JBIG2 for bilevel fax images, JPEG-2000 for still color images, and H.263+ and MPEG-4 for motion video. Finally, we describe some directions beyond the standards such as hybrid coding of graphics/photo images, MPEG-7 for multimedia metadata, and possible new technologies.

A block transform coder for arbitrarily shaped image segments

This paper describes a method for coding arbitrarily shaped image segments. The method uses an iterative technique based on the theory of successive projection onto convex sets to determine the best transform coefficients. It uses block transforms with frequency domain region-zeroing and space domain region-enforcing operations for effective coding of image segments of arbitrary shape. A major strength of this method is that it can be implemented in real-time using existing codec hardware at an insignificant additional cost.<<ETX>>

Interframe coding of videotelephone pictures

Television signals contain a great deal of frame-to-frame redundancy because picture areas are scanned in every frame whether they have changed or not. That portion of the signal describing stationary images need not be retransmitted in every frame if adequate memory is provided at the receiver. The signal describing moving images must be transmitted, of course, but it requires progressively less fidelity as the motion increases. Several techniques are described for obtaining efficient transmission by taking into account the similarity in the signal from frame to frame. Basic to most of these techniques is the need to separate the signal into segments that have changed significantly since the previous frame and ones that have not changed.

Interpolative vector quantization of color images

Interpolative vector quantization has been devised to alleviate the visible block structure of coded images plus the sensitive codebook problems produced by a simple vector quantizer. In addition, the problem of selecting color components for color picture vector quantization is discussed. Computer simulations demonstrate the success of this coding technique for color image compression at approximately 0.3 b/pel. Some background information on vector quantization is provided. >

Improvements in DCT-based video coding

We report on recent advances in traditional DCT based video coding at low bitrates. These improvements allow either an increase in coding efficiency or an increase in other functionalities. Our investigation is conducted within the framework of the ongoing work towards the MPEG-4 video standard. The ISO moving picture experts group (MPEG) is currently developing this standard after having completed the MPEG-1 and the MPEG-2 standards. The MPEG-4 video standard is addressing a number of content based as well as traditional functionalities. The development process consists of iterative refinement of the verification model via a set of well defined core experiments. Our first experiment is on improved coding efficiency of intra and uses DC and AC predictions and optimized scanning of DCT coefficients followed by a separate optimized variable length code table. Our second experiment is the study of bidirectional coding to allow additional functionality such as temporal scalability at low bit-rates. We present results of these experiments and summarize our findings.

On the applications of multimedia processing to communications

The challenge of multimedia processing is to provide services that seamlessly integrate text, sound, image, and video information and to do it in a way that preserves the ease of use and interactivity of conventional plain old telephone service (POTS) telephony. To achieve this goal, there are a number of technological problems that must be considered, including: compression and coding of multimedia signals, including algorithmic issues, standards issues, and transmission issues; synthesis and recognition of multimedia signals, including speech, images, handwriting, and text; organization, storage, and retrieval of multimedia signals, including the appropriate method and speed of delivery, resolution, and quality of service; access methods to the multimedia signal, including spoken natural language interfaces, agent interfaces, and media conversion tools; searching by text, speech, and image queries; browsing by accessing the text, by voice, or by indexed images. In each of these areas, a great deal of progress has been made in the past few years, driven in part by the relentless growth in multimedia personal computers and in part by the promise of broad-band access from the home and from wireless connections. Standards have also played a key role in driving new multimedia services, both on the POTS network and on the Internet. It is the purpose of this paper to review the status of the technology in each of the areas listed above and to illustrate current capabilities by describing several multimedia applications that have been implemented at AT&T Labs over the past several years.

Basics of stereoscopic video, new compression results with MPEG-2 and a proposal for MPEG-4

Abstract Many current as well as emerging applications in areas of entertainment, remote operations, manufacturing industry and medicine can benefit from the depth perception offered by stereoscopic video systems which employ two views of a scene imaged under the constraints imposed by the human visual system. Among the many challenges to be overcome for practical realization and widespread use of 3D/stereoscopic systems are good systems for 3D video capture, display and efficient techniques for digital compression of enormous amounts of data while maintaining compatibility with normal video decoding and display systems. After a brief introduction to the basics of 3D/stereo including issues of depth perception, imaging and display, we present a brief overview of portions of the MPEG-2 video standard that are relevant to our discussion on compression of stereoscopic video. Next, we outline the various approaches for compression of stereoscopic video and then focus on compatible stereoscopic video coding using MPEG-2 Temporal scalability concepts. Compatible coding employing two different types of prediction structures become potentially possible, disparity compensated prediction and combined disparity and motion compensated predictions. To further improve coding performance and display quality, gain and offset preprocessing for reducing mismatch between the two views forming stereoscopic video is considered. We then introduce the various considerations in coding of stereoscopic video at lower bit-rates for the ongoing MPEG-4 standard. A method is proposed that builds on the proven framework of MPEG-2 like coding but introduces additional coding flexibilities to achieve reasonable performance at lower bit-rates for MPEG-4. Next, results of experiments are presented for a variety of combinations of MPEG-2 based coding methods for the left and the right views while employing TV resolution video for a number of sequences and for various bit-rates. The combined disparity and motion-compensated prediction is found to offer the best performance among combinations tested. These results indicate that compression of both views of stereoscopic video of normal TV resolution appears feasible with good quality in a total of 6–8 Mbit/s. Further, results are presented at much lower bit-rates based on the coding method proposed for MPEG-4 on two long test sequences. We then discuss multi-viewpoint video applications, the ongoing efforts towards a multi-viewpoint profile in MPEG-2 and expected direction of multi-viewpoint video coding in MPEG-4.

Stereoscopic video compression using temporal scalability

Despite the fact that human ability to perceive a high degree of realism is directly related to our ability to perceive depth accurately in a scene, most of the commonly used imaging and display technologies are able to provide only a 2D rendering of the 3D real world. Many current as well as emerging applications in areas of entertainment, remote operations, industrial and medicine can benefit from the depth perception offered by stereoscopic video systems which employ two views of a scene imaged under the constraints imposed by human visual system. Among the many challenges to be overcome for practical realization and widespread use of 3D/stereoscopic systems are efficient techniques for digital compression of enormous amounts of data while maintaining compatibility with normal video decoding and display systems. After a brief discussion on the relationship of digital stereoscopic 3DTV with digital TV and HDTV, we present an overview of tools in the MPEG-2 video standard that are relevant to our discussion on compression of stereoscopic video, which is the main topic of this paper. Next, we determine ways in which temporal scalability concepts can be applied to exploit redundancies inherent between the two views of a scene comprising stereoscopic video. Due consideration is given to masking properties of stereoscopic vision to determine bandwidth partitioning between the two views to realize an efficient coding scheme while providing sufficient quality. Simulations are performed on stereoscopic video of normal TV resolution to compare the performance of the two temporal scalability configurations with each other and with the simulcast solution. Preliminary results are quite promising and indicate that the configuration that exploits motion and disparity compensation significantly outperforms the one that exploits disparity compensation alone. Compression of both views of stereo video of normal TV resolution appears feasible in a total of 8 or 9 Mbit/s. Finally, the implication of our results is discussed and potential directions for future research are identified.

An encoder-decoder texture replacement method with application to content-based movie coding

In this paper, we exploit the stylistic characteristics of high-quality entertainment movie sequences in terms of their textured background for coding purposes. More specifically, we propose a content-based coding method by texture replacement. At the encoder, texture is removed from selected regions of the original frames. The resulting frames with the texture removed and the parameters of the removed texture are then encoded. At the decoder, the boundaries of the regions without texture are identified and new texture, which is synthesized using the decoded texture parameters, is mapped onto these regions. Our experimental results confirm the main advantages of the proposed texture replacement method: significant bit rate reduction of the compressed movie sequences with the texture removed, and higher visual quality of the textured background regions in the decoded movie sequences with synthesized texture than that of the regions in the sequences simply encoded and decoded. Even more, our method can be applied as an overlay onto any standards-compliant coding system.