Johannes Ballé

Extended Texture Prediction for H.264/AVC Intra Coding

Efficient intra prediction is an important aspect of video coding with high compression efficiency. H.264/AVC applies directional prediction from neighboring pixels on an adjustable block size for local decorrelation. In this paper, we present an extended prediction scheme in the context of H.264/AVC that comprises two additional prediction methods exploiting self-similar properties of the encoded texture. A new macroblock type is implemented, allowing for flexible selection of the available prediction methods for sub-partitions of the macroblock. Depending on the content of the encoded video sequence, substantial gains in rate-distortion performance are achieved. The results may indicate directions towards an enhanced intra coding scheme with improved rate-distortion performance.

Multihypothesis prediction using decoder side-motion vector derivation in inter-frame video coding

In this paper, a multihypothesis prediction scheme for inter frame video coding is proposed. Using a template matching algorithm, motion vectors are derived at the decoder side instead of explicitly coding the motion vectors into the bitstream. Therefore, higher numbers of hypotheses can be used in the averaging process at no additional coding cost. The proposed scheme has been implemented into the H.264/AVC reference software. Simulation results show bitrate reductions compared to H.264/AVC of 7.7% on average for the tested video sequences. It is shown that part of the performance gain is due to rounding effects in H.264/AVC sub-pixel interpolation which can be exploited in the averaging calculation of the proposed multihypothesis prediction. Experiments with an improved interpolation filter for both reference scheme and the proposed scheme still yield bitrate reductions of 4.7% on average.

Models for Static and Dynamic Texture Synthesis in Image and Video Compression

In this paper, we investigate the use of linear, parametric models of static and dynamic texture in the context of conventional transform coding of images and video. We propose a hybrid approach incorporating both conventional transform coding and texture-specific methods for improvement of coding efficiency. Regarding static (i.e., purely spatial) texture, we show that Gaussian Markov random fields (GMRFs) can be used for analysis/synthesis of a certain class of texture. The properties of this model allow us to derive optimal methods for classification, analysis, quantization and synthesis. For video containing dynamic textures, a linear dynamic model can be derived from frames encoded in a conventional fashion. We show that after removing effects from camera motion, this model can be used to synthesize further frames. Beyond that, we show that using synthesized frames in an appropriate fashion for prediction leads to significant bitrate savings while preserving the same peak signal-to-noise ratio (PSNR) for sequences containing dynamic textures.

First Steps into Practical Engineering for Freshman Students Using MATLAB and LEGO Mindstorms Robots

Besides lectures on basic theoretical topics, contemporary teaching and learning concepts for first semester students give more and more consideration to practically motivated courses. In this context, a new first-year introductory course in practical engineering has been established in the first semester curriculum of Electrical Engineering at RWTH Aachen University, Germany. Based on a threefold learning concept, programming skills in MATLAB are taught to 309 students within a full-time block course laboratory. The students are encouraged to transfer known mathematical basics to program algorithms and real-world applications performed by 100 LEGO Mindstorms robots. A new MATLAB toolbox and twofold project tasks have been developed for this purpose by a small team of supervisors. The students are supervised by over 60 tutors at 23 institutes, and are encouraged to create their own robotics applications. We describe how the laboratory motivates the students to act and think like engineers and to solve real-world issues with limited resources. The evaluation results show that the proposed practical course concept successfully boosts students’ motivation, advances their programming skills, and encourages the peer learning process.

Median trilateral loop filter for depth map video coding

Emerging extensions to conventional stereo video technologies like 3D Video require to add depth information to 2D video data. This supplementary data needs to be coded efficiently and transmitted to the receiver where arbitrary viewpoints are generated by using this additional information. The depth maps are characterized by piecewise smooth regions, which are bounded by sharp edges describing depth discontinuities along object boundaries. Preserving these characteristics and especially depth discontinuities is a crucial requirement for depth map coding. When coding depth maps by means of a conventional hybrid video coder, ringing artifacts are introduced along the sharp edges and result in quality degradation when using the reconstructed depth maps for view synthesis. To reduce these ringing artifacts and also to better align object boundaries in video and depth data, a new in-loop filter is proposed, which reconstructs the described characteristics of depth maps.

Efficient shift-variant image restoration using deformable filtering (Part II): PSF field estimation

AbstractWe present a two-step technique for estimating the point spread function (PSF) field from a single star field image affected by shift-variant (SV) blur. The first step estimates the best-fitting PSF for each block of an overlapping block grid. We propose a local image model consisting of a pattern (the PSF) being replicated at arbitrary locations and with arbitrary weights. We follow an efficient alternate marginal optimization approach for estimating (1) the most likely pattern, and (2) the locations where it appears in the block, with sub-pixel accuracy. The second step uses linear dimensionality reduction and nonlinear spatial filtering for estimating the entire PSF field from the grid of local PSF estimates. We simulate SV blur on realistic synthetic star fields to assess the accuracy of the method for this kind of images, for different blurs, star densities, and Poisson counts. The results indicate a moderately low error and very robust behavior against noise and artifacts. We also apply our method to real astronomical images, and demonstrate that the method provides relevant information about the underlying structure of the actual telescope and atmosphere PSF fields. We use a variant of the method proposed in Part I to compensate for the observed blur.

Subjective evaluation of texture similarity metrics for compression applications

The paper summarizes the results of an experimental subjective evaluation of texture similarity metrics with 25 test subjects. The compared metrics comprise the frequency-weighted log-spectral and Itakura distances, as well as a set of metrics based on an overcomplete Gabor-like filterbank - the STSIM as well as two new ones. The test set consists of 30 synthetic GMRF texture pairs. It turns out that all metrics perform well, but the weighted log-spectral distance tends to outperform the filterbank-based metrics.

Improved entropy coding for component-based image coding

In this paper, we improve on our previous work regarding component-based image coding, a hybrid transform-based/perceptual image coding scheme based on a decomposition of the image into structure and texture characterized by a Gaussian Markov random field. The 2D Itakura distance allows us to evaluate the performance of our texture model in terms of rate vs. distortion. A minimal quantization step size for near-lossless coding of model parameters is determined. Furthermore, we show that texture contrast can be efficiently coded using transform-based techniques.

Component-based image coding using non-local means filtering and an autoregressive texture model

While noise is usually regarded as a problem of the image formation process, we observe that it is also frequently part of natural texture. In this paper, we present a concept for improved compression of noisy texture in natural images. Since noise is problematic to decorrelation-based compression methods, we propose to perform image decomposition by denoising, followed by separate compression of the components. The denoised component is encoded using conventional methods, while the texture component is compressed by encoding parameters of a texture model. It turns out that at similar bit rates, our method can improve visual quality.

Implementing Grid Networks in the CE Domain

Today, grid networks are by far the most popular technique for distributing large media files via the Internet. The technology benefits from the free bandwidth donated by every node in the network, resulting in a near zero-cost distribution channel for content providers. However, current implementations have several shortcomings preventing broad establishment of the technology in the CE domain. This paper describes an end-to-end media distribution infrastructure that addresses these problems by delegating all tasks from downloading to rendering the content to embedded devices, which are easy to use, cost-effective, and eliminate the need for a PC