André Kaup

Object-based texture coding of moving video in MPEG-4

This paper describes some of the most promising segment-based coding techniques which have been investigated in the course of the MPEG-4 standardization process. Padding methods aim at extending arbitrarily shaped image segments to a regular block grid such that common hybrid block-based coding techniques can be applied. A simple and efficient padding technique employing low-pass extrapolation is outlined which yields a signal extension with high energy concentration in the low-frequency area. Simulations indicate that this method is well suited for block-based video coding, and clearly outperforms other low-complexity extrapolation methods with respect to coding efficiency. In contrast to padding techniques, shape-adaptive methods take advantage of the shape information available at the decoder side. A well-known representative of this class is the SA-DCT. However, having been primarily designed for intraframe coding, it is shown that the transform is suboptimal when applied to interframe coding. Using a suitable covariance model, it is demonstrated that a rescaled, orthonormalized transform much closer approximates the optimal shape-adaptive eigentransform of motion-compensated frame difference images. Rate distortion curves verify that orthonormalization improves coding efficiency in interframe coding by up to 2 dB while not adding to complexity. In a comparison, it is finally shown that extrapolation and SA-DCT perform very closely in the case of low data rates, while there is a clear advantage for the shape-adaptive transform in the case of high-quality video coding.

Coding of segmented images using shape-independent basis functions

An important issue in object-based image coding is the efficient description of image segments having an arbitrary shape. This paper outlines a solution to this task using basic principles of transform image coding that are generalized for the case of arbitrarily shaped image segments. The texture inside each region is successively approximated using two-dimensional (2-D) shape-independent basis functions defined on a rectangle circumscribing the given image segment. The resulting texture description exhibits a high energy compactness and is well suited for low bit rate image coding. Unlike other approaches aimed at segment-oriented image coding, the proposed concept does not couple texture description with contour coding. The computational load is kept low, especially for the decoding process.

Global motion estimation in image sequences using robust motion vector field segmentation

In this paper we propose an algorithm for the purpose of video indexing which estimates the global motion caused by camera movement. Since motion estimation is a computationally expensive process, we are using information already computed in block based video encoders such as the displacement vectors of the blocks and the computed SAD values of the blockmatching process. With this approach we could significantly speed up the determination of the parameters of a chosen global motion model. The accuracy of the algorithm is improved further by introducing a reliability measure. To achieve a robust computation with respect to object motion, an effective and straightforward segmentation of this vector field is implemented. Finally, the modeling of the global motion is evaluated using MPEG-7 test data.

Complexity and PSNR comparison of several fast motion estimation algorithms for MPEG-4

A complexity and visual quality analysis of several fast motion estimation (ME) algorithms for the emerging MPEG-4 standard was performed as a basis for HW/SW partitioning for VLSI implementation of a portable multimedia terminal. While the computational complexity for the ME of previously standardized video coding schemes was predictable over time, the support of arbitrarily shaped visual objects (VO), various coding options within MPEG-4 as well as content dependent complexity (caused e.g. by summation truncation for SAD) introduce now content (and therefore time) dependent computational requirements, which cant be determined analytically. Therefore a new time dependent complexity analysis method, based on statistical analysis of memory access bandwidth, arithmetic and control instruction counts utilized by a real processor, was developed and applied. Fast ME algorithms can be classified into search area subsampling, pel decimation, feature matching, adaptive hierarchical ME and simplified distance criteria. Several specific implementations of algorithms belonging to these classes are compared in terms of complexity and PSNR to ME algorithms for arbitrarily and rectangular shaped VOs. It is shown that the average macroblock (MB) computational complexity per arbitrary shaped P-VOP (video object plane) depicts a significant variation over time for the different motion estimation algorithms. These results indicate that theoretical estimations and the number of MBs per VOP are of limited applicability as approximation for computational complexity over time, which is required e.g. for average system load specification (in contrast to worst case specification), for real-time processor task scheduling, and for Quality of Service guarantees of several VOs.

Adaptive low-pass extrapolation for object-based texture coding of moving video

The extension of conventional video coding schemes by additional content-based functionalities is a key issue in MPEG-4 standardization. To this end, suitable techniques for coding texture information of arbitrary shaped image segments have to be developed. In this paper, a new approach for coding of such image segments is presented which is based on signal extrapolation. In contrast to other extrapolation approaches described in the literature, the proposed method is computationally simple and specifically adapted to the needs of real-time video coding. The extrapolation works in the spatial domain and yields a signal with high energy concentration in the low frequency area. Restricting the extrapolation to a block grid allows straight forward extension of common block-based video coding techniques to coding of arbitrarily shaped image objects.

Quality improvement of low-data-rate compressed video signals by pre- and postprocessing

In this paper a technique for improving the image quality of block-based video coders is presented which combines the following pre- and postprocessing steps: Before the coding process, the image sharpness is enhanced by applying a special prefilter to the original image. By this means a possible degradation of the image sharpness by the coder is compensated. The decoder output is smoothed by a lowpass filter which reduces artifacts such as block discontinuities. Secondly, the quantization error of the frequency DCT coefficients is reduced by predicting AC coefficients based on the mean values of the neighboring image blocks. This step improves the luminance function within the blocks. While the original sequences show coarse blocking patterns, the images processed by this technique look smooth on the overall with well-preserved local edge structures. The tool supports all standardized DCT codecs, such as MPEG-1/2 or H.261/3.

Adaptive constrained least squares restoration for removal of blocking artifacts in low bit rate video coding

For high compression ratios current video coding standards produce noticeable blocking and ringing noise due to a rigid block structure and coarse quantization. We propose a new method for reduction of these coding artifacts based on spatially adaptive constrained least squares restoration. The proposal is numerically simple and yields visually convincing results for intra as well as inter coded images. As a post-processing technique it is compatible to all existing image and video coding standards.

Performance and complexity analysis of rate-constrained motion estimation in MPEG-4

Motion estimation constitutes a critical part in video coding since it is computationally expensive and has a substantial impact on the resulting image quality. A very efficient method to estimate motion is through adding a rate constraint for coding of the motion vectors into the estimation process. In this paper we analyze the performance of this approach in the emerging MPEG-4 standard, considering frame- as well as object-based video sequences. The investigations show that rate constrained motion estimation generally offers significant gains in coding efficiency which, however, depend on the shape of the object coded. Complexity measurements further show that a rate constraint imposes only a marginal computational overhead, thus making this approach a useful amendment for an MPEG-4 coder.

Error concealment for SNR scalable video coding in wireless communication

A mandatory requirement for future wireless multimedia communication is the availability of error resilient media codecs. This work discusses error detection and concealment techniques for a layered scalable video decoder, based on the SNR scalability option of the H.263 standard. A concealment method is proposed which uses error frequency and error location statistics for efficient hiding of transmission errors. The designed decoder allows error robust decoding with acceptable image quality even for highly corrupted video sequences up to bit error rates of 10-3, optimizing the image quality specifically for two-layer coded sequences. Simulation results show that scalable video coding outperforms single-layer coding under typical wireless conditions even if no priority or increased protection is applied to the base layer. The proposed two- layer concealment method yields a consistent improvement of up to 5 dB in image quality.

Visual feature discrimination versus compression ratio for polygonal shape descriptors

In the last decade several methods for low level indexing of visual features appeared. Most often these were evaluated with respect to their discrimination power using measures like precision and recall. Accordingly, the targeted application was indexing of visual data within databases. During the standardization process of MPEG-7 the view on indexing of visual data changed, taking also communication aspects into account where coding efficiency is important. Even if the descriptors used for indexing are small compared to the size of images, it is recognized that there can be several descriptors linked to an image, characterizing different features and regions. Beside the importance of a small memory footprint for the transmission of the descriptor and the memory footprint in a database, eventually the search and filtering can be sped up by reducing the dimensionality of the descriptor if the metric of the matching can be adjusted. Based on a polygon shape descriptor presented for MPEG-7 this paper compares the discrimination power versus memory consumption of the descriptor. Different methods based on quantization are presented and their effect on the retrieval performance are measured. Finally an optimized computation of the descriptor is presented.