Adriana Dumitras

A texture replacement method at the encoder for bit-rate reduction of compressed video

We propose a method for texture replacement in video sequences. Our method, which is applied at the encoder side, consists of removal of texture from selected regions of the original frames, synthesis of new texture, and mapping of the new texture back onto the segmented regions. The texture removal stage employs highly effective color-based angular maps. The texture analysis and texture synthesis stages make use of steerable pyramids. The latter stage also employs constraints that are derived using a vocabulary and grammar for color pattern similarity evaluation that have been introduced previously. Because they have different characteristics than those of the original textures, the synthesized textures can be coded more effectively. Consequently and most importantly, significantly reduced bit rates of the compressed video sequences with the texture replaced are obtained as compared to those of the original sequences. Moreover, because the synthesized textures have similar perceptual characteristics to those of the original textures, the video sequences with the texture replaced are also visually similar to the original sequences. Even more, because it is performed at the encoder and it does not have any impact on the decoder, our texture replacement method is cost effective. We illustrate its performance and computational efficiency using movie sequences.

I/P/B frame type decision by collinearity of displacements

State-of-art encoders in real life applications typically select the number of B frames to be coded between each I or P frame to be equal to one or two by default. The few research works that have addressed the problem of encoder optimization by frame type decision rely on measures of motion magnitude and amount to select the number of B frames. In contrast to such solutions, in this paper we advocate the idea of motion similarity in terms of speed and direction as the basis for deciding how many B frames to encode between any two stored frames. Such similarity is evaluated by the collinearity of the displacements in successive frames. Experimental results using the proposed decision method integrated in our H.264 compliant codec show that the bit rates achieved in the compressed bitstreams are optimal and near-optimal for the tested sequences, and are lower by up to 26% than those obtained using the default number of B frames of typical encoders. Furthermore, such bit rate savings are obtained with no subjective loss in the visual quality of the decoded sequences. Last but not least, our method provides a simple form of temporal scalability and can be employed in any coding system that makes use of I, P and B frames.

An automatic method for unequal and omni-directional anisotropic diffusion filtering of video sequences

In this paper, we propose a method for automatic, unequal and omni-directional anisotropic diffusion filtering of video sequences. Our method, which consists of automatic foreground/background discrimination using the color characteristics of the human face and skin, unequal filtering of the determined regions and then composition of the filtered regions into the final frames, is applied prior to encoding the video sequences. In addition to yielding higher visual quality of the decoded sequences than that of the decoded sequences with equal filtering over the entire frame, our method is robust with respect to skin tone and lighting variations, has low complexity and requires no calibration. We illustrate the main advantages of our method using videoconferencing sequences.

A look-ahead method for pan and zoom detection in video sequences using block-based motion vectors in polar coordinates

Identifying pans and zooms in video sequences allows parameter switching for more effective encoding in standards such as H.264, and retrieval of documentary movies in video databases. In this work we propose a simple yet effective look-ahead method for pan and zoom detection. Our method which evaluates the orientation of block-based motion vectors to identify global pan and zoom motion departs from traditional global motion analysis works. Experimental results show that our method is very effective and computationally efficient.

Enhancement of direct mode selection in B pictures for bit rate reduction of compressed video sequences

In this paper, we propose a method for direct mode enhancement in B pictures in the framework of H.264 (MPEG-4/Part 10), which employs reduction of outliers, clustering of the Lagrangian values and specification of smaller values of the Lagrangian multiplier in the rate-distortion optimization for mode selection. Experimental results using high quality video sequences show that bit rate reduction and no subjective visual loss in the decoded pictures are obtained using our method. Moreover, we verify that our method achieves further bit rate reduction without introducing visible distortion in the decoded sequences when other solutions would introduce inadmissible artifacts. Last but not least, the proposed method is a simple and useful add-on, which is applicable in any video coding system that employs rate-distortion optimization for coding mode selection.

Editorial: video analysis and coding for robust transmission

Increasing heterogeneity of networks and diversity of user capabilities have determined and sustained a strong interest in robust coding of visual content and flexible adaptation of the bitstreams to network and user conditions. As a result, several methods for robust coding and transmission have been proposed that include multiple description coding, motioncompensated subband video coding, joint source-channel coding, integrated compression and error control, and adaptation/transcoding solutions. These typically increase transmission robustness and network and user awareness by using scalability, error resilience, and adaptivity at little or sometimes no extra cost in coding efficiency. However, the performance of these methods is affected by the diversity of, and complex interactions within, the visual content. Analysis methods can improve the performance of robust methods for coding and transmission by providing solutions to account for vastly different characteristics of the visual content and complex interactions among data components, to achieve optimal or near-optimal robust solutions. Among several benefits, the application of visual analysis methods within robust coding and transmission frameworks such as those mentioned earlier yields content-aware error resilient solutions and improves prioritization of the visual content for coding and transmission. This special issue focuses on the seamless integration of visual analysis methods in, or joint design with, robust compression and transmission solutions. The special issue consists of three sections that address robust video coding architectures and configurations, robust entropy coding methods, and quality issues related to robust coding, respectively: