Tobias Tröger

Color constancy and non-uniform illumination: Can existing algorithms work?

The color and distribution of illuminants can significantly alter the appearance of a scene. The goal of color constancy (CC) is to remove the color bias introduced by the illuminants. Most existing CC algorithms assume a uniformly illuminated scene. However, more often than not, this assumption is an insufficient approximation of real-world illumination conditions (multiple light sources, shadows, interreflections, etc.). Thus, illumination should be locally determined, taking under consideration that multiple illuminants may be present. In this paper we investigate the suitability of adapting 5 state-of-the-art color constancy methods so that they can be used for local illuminant estimation. Given an arbitrary image, we segment it into superpixels of approximately similar color. Each of the methods is applied independently on every superpixel. For improved accuracy, these independent estimates are combined into a single illuminant-color value per superpixel. We evaluated different fusion methodologies. Our experiments indicate that the best performance is obtained by fusion strategies that combine the outputs of the estimators using regression.

4D Scalable Multi-View Video Coding Using Disparity Compensated View Filtering and Motion Compensated Temporal Filtering

In this paper, a novel framework for scalable multi-view video coding is described. A well known wavelet based scalable coding scheme for single-view video sequences has been adopted and extended to match the specific needs of scalable multi-view video coding. Motion compensated temporal filtering (MCTF) is applied to each video sequence of each camera. The use of a wavelet lifting structure guarantees perfect invertibility of this step, and as a consequence of its open-loop architecture, SNR and temporal scalability are attained. Correlations between the temporal subbands of adjacent cameras are reduced by a novel disparity compensated view filtering (DCVF), method which is also lifting based and open-loop to enable view scalability. Spatial scalability and entropy coding are achieved by the JPEG2000 spatial wavelet transform and EBCOT coding, respectively. Rate allocation along the temporal-view-filtered subbands is done by means of an RD-optimal algorithm. Experimental results show the high scaling capability in terms of SNR, temporal and view scalability

Inter-Sequence Error Concealment Techniques for Multi-Broadcast TV Reception

Mobile reception of digital TV signals often suffers from lost slices due to suboptimal channel characteristics. In this contribution we propose novel inter-sequence error concealment techniques which reconstruct lost blocks of distorted high-resolution TV signals by inserting error-free blocks of low-resolution reference signals. The proposed methods are well-suited for future automotive multi-broadcast receivers. As the reference signals are typically delayed, scaled and possibly cropped versions of the distorted TV signals, spatio-temporal image alignment is a crucial point. We first focus on pixel-based alignment utilizing a numerical optimization scheme. To reduce complexity a feature-based approach is introduced alternatively taking advantage of the scale-invariant feature transform. Simulations show that the proposed techniques can achieve supreme reconstruction quality outperforming conventional techniques by up to 15.0 dB . The pixel-based approach mostly reaches the maximum reconstruction quality for both raw and compressed video signals. Feature-based concealment performs slightly worse but reduces the computation time by a factor of up to 36.6 at a maximum loss of 1.0 dB. A novel hybrid concealment technique combining the proposed methods with a temporal approach further increases the reconstruction quality. Applying one simple and one sophisticated mode selection scheme, a gain of 3.2 dB and 4.6 dB can be achieved, respectively.

Fast and robust spatio-temporal image alignment for inter-sequence error concealment

Terrestrial broadcasting of digital TV is often error-prone due to suboptimal channels. Inter-sequence error concealment is a novel technique to reconstruct lost image blocks of video signals by inserting error-free blocks of reference signals. Considering multi-broadcast receivers, reference signals are typically scaled, cropped and delayed. Spatio-temporal image alignment therefore is the crucial point of inter-sequence error concealment. A well-known intensity-based approach utilizing a numerical optimization technique shows supreme reconstruction quality outperforming state-of-the-art concealment methods by up to 15 dB PSNRY. In this paper, we present a novel feature-based approach which significantly reduces complexity. By using scale-invariant features for image alignment, the computation time can be reduced by a factor of up to 32.4. As reconstruction quality is almost kept constant, robustness of our algorithm is high. The maximum loss is 0.8 dB in terms of PSNRY.

Mixed-resolution view synthesis using non-local means refined image merging

Synthesizing novel views from originally available camera perspectives via depth maps is a key issue in the 3D video domain. Up to now, several high-resolution cameras are needed to obtain high-quality intermediate synthesized views. One possibility to reduce costs with regard to the used camera array is to replace some cameras by low-resolution cameras, which are cheaper on the one hand, but provide a much poorer image quality on the other hand. Unfortunately, some of the information inside the desired intermediate view may only be available in the low-resolution reference. Thus, the image quality of the low-resolution reference has a big influence on the visual quality of the synthesized view. This paper proposes a postprocessing step for the synthesized view, based on the non-local means algorithm. Thereby, all areas inserted from the low-resolution reference get efficiently adapted to their high-resolution environment. It is shown, that the non-local means refined image merging leads to a PSNR gain of up to 0.90 dB compared to an unrefined mixed-resolution setup. The approach can be easily extended to a hole-filling algorithm and yields a PSNR gain of up to 0.81 dB for hole areas compared to a reference hole-filling algorithm. The subjective image quality also increases convincingly in both applications.

Low-complexity inter-sequence error concealment based on scale-invariant feature transform

Inter-sequence error concealment defines a novel class of concealment techniques which utilize two or more representations of a particular video sequence for image reconstruction. An image-based approach introduced recently provides superior reconstruction quality of concealed image parts and outperforms well-known intra methods by up to 15 dB PSNRY. However, computational complexity is high due to a numerical solution of the image registration problem. In this paper, we propose a feature-based technique for image registration which is included in the concept of inter-sequence error concealment. Based on the determination of scale-invariant features, the processing time is reduced on average by a factor of up to 28. At the same time, the loss in terms of objective image quality compared to the image-based approach is less than 1 dB PSNRY.

Efficient lossless coding of highpass bands from block-based motion compensated wavelet lifting using JPEG 2000

Lossless image coding is a crucial task especially in the medical area, e.g., for volumes from Computed Tomography or Magnetic Resonance Tomography. Besides lossless coding, compensated wavelet lifting offers a scalable representation of such huge volumes. While compensation methods increase the details in the lowpass band, they also vary the characteristics of the wavelet coefficients, so an adaption of the coefficient coder should be considered. We propose a simple invertible extension for JPEG 2000 that can reduce the filesize for lossless coding of the highpass band by 0.8% on average with peak rate saving of 1.1%.

Difference image extrapolation for spectral completion in inter-sequence error concealment

Mobile reception of digital TV often suffers from lost blocks and slices due to suboptimal channels. Inter-sequence error concealment reconstructs lost image blocks of distorted high-resolution TV signals by inserting corresponding error-free blocks of low-resolution reference signals. It is well-suited for automotive multi-broadcast receivers and can outperform state-of-the-art methods by up to 15 dB PSNRY. In this contribution, a novel spectral completion scheme is proposed which robustly estimates missing spatial frequencies of reconstructed blocks by extrapolation of difference images. Simulation results show that the proposed scheme can increase the reconstruction quality of inter-sequence error concealment by up to 3.2 dB. Also, the subjective quality is enhanced by significantly reducing blurring artifacts.

Improved mode selection in hybrid error concealment for multi-broadcast-reception

Digital TV signals are often degraded by lost macro blocks or slices if received with mobile devices. Recently, we introduced a technique for inter-sequence error concealment in multi-broadcast scenarios which reconstruct lost image blocks of video signals by inserting error-free blocks of corresponding reference signals. The algorithm outperforms state-of-the-art techniques by up to 15 dB PSNRY. The quality of reconstructed image blocks can he further enhanced by combining inter-sequence error concealment with a temporal concealment approach as we showed in 2009. To enhance the robustness of this hybrid error concealment, we propose a novel metric for mode selection in this paper which takes into account local image statistics. While keeping the complexity of hybrid error concealment nearly constant, a mean gain of up to 2.1 dB and a peak gain of up to 10.1 dB in terms of PSNRY can be achieved additionally.

Spatially refined inter-sequence error concealment for a multi-broadcast receiver using frequency selective approximation

Mobile reception of digital TV often suffers from severe signal degradations. Inter-sequence error concealment reconstructs lost image blocks of a distorted high-resolution TV signal by inserting corresponding error-free blocks from a low-resolution reference TV signal. It is well-suited for application in future automotive multi-broadcast receivers and can outperform state-of-the-art methods by up to 15 dB PSNRY depending on the quality of the reference signal. In this contribution, we show that inter-sequence error concealment can be improved by approximating inserted blocks jointly with neighboring pixels according to a well-known frequency selective method. The reconstruction quality can be significantly increased especially in case of low-bitrate reference signals. Inserted blocks can be further refined also for high bitrates. On average, a gain of 1.7 dB PSNRY can be achieved. The peak gain which is evaluated on frame basis even reaches 5.6 dB PSNRY.