Gauthier Lafruit

Cross-Based Local Stereo Matching Using Orthogonal Integral Images

We propose an area-based local stereo matching algorithm for accurate disparity estimation across all image regions. A well-known challenge to local stereo methods is to decide an appropriate support window for the pixel under consideration, adapting the window shape or the pixelwise support weight to the underlying scene structures. Our stereo method tackles this problem with two key contributions. First, for each anchor pixel an upright cross local support skeleton is adaptively constructed, with four varying arm lengths decided on color similarity and connectivity constraints. Second, given the local cross-decision results, we dynamically construct a shape-adaptive full support region on the fly, merging horizontal segments of the crosses in the vertical neighborhood. Approximating image structures accurately, the proposed method is among the best performing local stereo methods according to the benchmark Middlebury stereo evaluation. Additionally, it reduces memory consumption significantly thanks to our compact local cross representation. To accelerate matching cost aggregation performed in an arbitrarily shaped 2-D region, we also propose an orthogonal integral image technique, yielding a speedup factor of 5-15 over the straightforward integration.

Optimal memory organization for scalable texture codecs in MPEG-4

This paper addresses the problem of minimizing memory size and memory accesses in multiresolution texture coding architectures for discrete cosine transform (DCT) and wavelet-based schemes used, for example, in virtual-world walk-throughs or facial animation scenes of an MPEG-4 system. The problem of minimizing the memory cost is important since memory accesses, memory bandwidth limitations, and in general the correct handling of the data flows have become the true critical issues in designing high-speed and low-power video-processing architectures and in efficiently using multimedia processors. For instance, the straightforward implementation of a multiresolution texture codec typically needs an extra memory buffer of the same size as the image to be encoded/decoded. We propose a new calculation schedule that reduces this buffer memory size with up to two orders of magnitude, while still ensuring a number of external (off-chip) memory accesses that is very close to the theoretical minimum. The analysis is generic and is therefore useful for both wavelet and multiresolution DCT codecs.

Scheduling and Resource Allocation for SVC Streaming Over OFDM Downlink Systems

We consider the problem of scheduling and resource allocation for multiuser video streaming over downlink orthogonal frequency division multiplexing (OFDM) channels. The video streams are precoded using the scalable video coding (SVC) scheme that offers both quality and temporal scalabilities. The OFDM technology provides the flexibility of resource allocation in terms of time, frequency, and power. We propose a gradient-based scheduling and resource allocation algorithm, which prioritizes the transmissions of different users by considering video contents, deadline requirements, and transmission history. Simulation results show that the proposed algorithm outperforms the content-blind and deadline-blind algorithms with a gain of as much as 6 dB in terms of average PSNR when the network is congested.

SIFER: Scale-Invariant Feature Detector with Error Resilience

We present a new method to extract scale-invariant features from an image by using a Cosine Modulated Gaussian (CM-Gaussian) filter. Its balanced scale-space atom with minimal spread in scale and space leads to an outstanding scale-invariant feature detection quality, albeit at reduced planar rotational invariance. Both sharp and distributed features like corners and blobs are reliably detected, irrespective of various image artifacts and camera parameter variations, except for planar rotation. The CM-Gaussian filters are approximated with the sum of exponentials as a single, fixed-length filter and equal approximation error over all scales, providing constant-time, low-cost image filtering implementations. The approximation error of the corresponding digital signal processing is below the noise threshold. It is scalable with the filter order, providing many quality-complexity trade-off working points. We validate the efficiency of the proposed feature detection algorithm on image registration applications over a wide range of testbench conditions.

Real-time accurate stereo with bitwise fast voting on CUDA

This paper proposes a real-time design for accurate stereo matching on Compute Unified Device Architecture (CUDA). We adopt a leading local algorithm for its high data parallelism. A GPU-oriented bitwise fast voting method is proposed to effectively improve the matching accuracy, which is enormously faster than the histogram-based approach. The whole algorithm is parallelized on CUDA at a fine granularity, efficiently exploiting the computing resources of GPUs. On-chip shared memory is utilized to alleviate the latency of memory accesses. Compared to the CPU counterpart, our design attains a speedup factor of 52. With high matching accuracy, the proposed design is still among the fastest stereo methods on GPUs. The advantages of speed and accuracy advocate our design for practical applications such as robotics systems and multiview teleconferencing.

An efficient VLSI architecture for 2-D wavelet image coding with novel image scan

A folded very large scale integration (VLSI) architecture is presented for the implementation of the two-dimensional discrete wavelet transform, without constraints on the choice of the wavelet-filter bank. The proposed architecture is dedicated to flexible block-oriented image processing, such as adaptive vector quantization used in wavelet image coding. We show that reading the image along a two-dimensional (2-D) pseudo-fractal scan creates a very modular and regular data flow and, therefore, considerably reduces the folding complexity and memory requirements for VLSI implementation. This leads to significant area savings for on-chip storage (up to a factor of two) and reduces the power consumption. Furthermore, data scheduling and memory management remain very simple. The end result is an efficient VLSI implementation with a reduced area cost compared to the conventional approaches, reading the input data line by line.

Real-Time and Accurate Stereo: A Scalable Approach With Bitwise Fast Voting on CUDA

This paper proposes a real-time design for accurate stereo matching on compute unified device architecture (CUDA). We present a leading local algorithm and then accelerate it by parallel computing. High matching accuracy is achieved by cost aggregation over shape-adaptive support regions and disparity refinement using reliable initial estimates. A novel sample-and-restore scheme is proposed to make the algorithm scalable, capable of attaining several times speedup at the expense of minor accuracy degradation. The refinement and the restoration are jointly realized by a local voting method. To accelerate the voting on CUDA, a graphics processing unit (GPU)-oriented bitwise fast voting method is proposed, faster than the traditional histogram-based approach with two orders of magnitude. The whole algorithm is parallelized on CUDA at a fine granularity, efficiently exploiting the computing resources of GPUs. Our design is among the fastest stereo matching methods on GPUs. Evaluated in the Middlebury stereo benchmark, the proposed design produces the most accurate results among the real-time methods. The advantages of speed, accuracy, and desirable scalability advocate our design for practical applications such as robotics systems and multiview teleconferencing.

Initial memory complexity analysis of the AVC codec

The Advanced Video Codec (AVC), currently being defined in a joined standardisation effort of ISO/IEC MPEG and ITU-T VCEG, aims at enhanced compression efficiency and network friendliness. To achieve these goals, a motion compensated hybrid DCT algorithm is introduced using advanced and complicated compression tools. As video coding is typically a data dominated process, we quantify the complexity cost in a memory centric way. The AVC codec is characterised by a large memory footprint and increased data transfer rate (an order of magnitude for the encoder) compared to previous video coding standards. The motion estimation/compensation are the initial implementation bottlenecks.

3D computational graceful degradation

With new multimedia standards, such as MPEG-4, media can progressively be transmitted at different levels of detail, which allows dynamic adaptation to the available network bandwidth. We present a new technique, 3D Computational Graceful Degradation (CGD), that exploits this incremental coding/decoding process to constrain the terminals processing requirements to a predefined level, independently of the degree of complexity of the incoming data. Our attention is directed at 3D scenes, for which the variability of content complexity can range over several orders of magnitude. We provide evidence that the load of the 3D decoding and rendering modules can predictively be estimated and controlled, using a limited amount of statistical measures of the incoming 3D data.

Anisotropic local high-confidence voting for accurate stereo correspondence

We present a local area-based, discontinuity-preserving stereo matching algorithm that achieves high quality results near depth discontinuities as well as in homogeneous regions. To address the well-known challenge of defining appropriate support windows for local stereo methods, we use the anisotropic Local Polynomial Approximation (LPA) - Intersection of Confidence Intervals (ICI) technique. It can adaptively select a nearoptimal anisotropic local neighborhood for each pixel in the image. Leveraging this robust pixel-wise shape-adaptive support window, the proposed stereo method performs a novel matching cost aggregation step and an effective disparity refinement scheme entirely within a local high-confidence voting framework. Evaluation using the benchmark Middlebury stereo database shows that our method outperforms other local stereo methods, and it is even better than some algorithms using advanced but computationally complicated global optimization techniques.