Pierre Greisen

Computational stereo camera system with programmable control loop

Stereoscopic 3D has gained significant importance in the entertainment industry. However, production of high quality stereoscopic content is still a challenging art that requires mastering the complex interplay of human perception, 3D display properties, and artistic intent. In this paper, we present a computational stereo camera system that closes the control loop from capture and analysis to automatic adjustment of physical parameters. Intuitive interaction metaphors are developed that replace cumbersome handling of rig parameters using a touch screen interface with 3D visualization. Our system is designed to make stereoscopic 3D production as easy, intuitive, flexible, and reliable as possible. Captured signals are processed and analyzed in real-time on a stream processor. Stereoscopy and user settings define programmable control functionalities, which are executed in real-time on a control processor. Computational power and flexibility is enabled by a dedicated software and hardware architecture. We show that even traditionally difficult shots can be easily captured using our system.

An FPGA-based processing pipeline for high- definition stereo video

This paper presents a real-time processing platform for high-definition stereo video. The system is capable to process stereo video streams at resolutions up to 1, 920 × 1, 080 at 30 frames per second (1080p30). In the hybrid FPGA-GPU-CPU system, a high-density FPGA is used not only to perform the low-level image processing tasks such as color interpolation and cross-image color correction, but also to carry out radial undistortion, image rectification, and disparity estimation. We show how the corresponding algorithms can be implemented very efficiently in programmable hardware, relieving the GPU from the burden of these tasks. Our FPGA implementation results are compared with corresponding GPU implementations and with other implementations reported in the literature.

Matching pursuit: Evaluation and implementatio for LTE channel estimation

The emerging research field of compressed sensing (CS) promises better signal reconstruction out of fewer measurements if a sparse representation of the signal exists. Since wireless broadband channels often exhibit a sparse impulse response, CS reconstruction algorithms were proposed for channel estimation. In this paper, a hardware architecture for channel estimation using the matching pursuit algorithm is presented. The reference design targets the 3GPP LTE standard with a channel bandwidth of up to 20 MHz. Achievable performance gains over least squares channel estimation are illustrated by means of simulations. The costs in terms of chip area and reconstruction time for 180 nm CMOS technology are presented together with an analysis of the tradeoff between hardware complexity and reconstruction performance.

Automatic View Synthesis by Image-Domain-Warping

Today, stereoscopic 3D (S3D) cinema is already mainstream, and almost all new display devices for the home support S3D content. S3D distribution infrastructure to the home is already established partly in the form of 3D Blu-ray discs, video on demand services, or television channels. The necessity to wear glasses is, however, often considered as an obstacle, which hinders broader acceptance of this technology in the home. Multiviewautostereoscopic displays enable a glasses free perception of S3D content for several observers simultaneously, and support head motion parallax in a limited range. To support multiviewautostereoscopic displays in an already established S3D distribution infrastructure, a synthesis of new views from S3D video is needed. In this paper, a view synthesis method based on image-domain-warping (IDW) is presented that automatically synthesizes new views directly from S3D video and functions completely. IDW relies on an automatic and robust estimation of sparse disparities and image saliency information, and enforces target disparities in synthesized images using an image warping framework. Two configurations of the view synthesizer in the scope of a transmission and view synthesis framework are analyzed and evaluated. A transmission and view synthesis system that uses IDW is recently submitted to MPEGs call for proposals on 3D video technology, where it is ranked among the four best performing proposals.

Implementation of greedy algorithms for LTE sparse channel estimation

Broadband wireless systems often operate under channel conditions that are characterized by a sparse channel impulse response. When the amount of training is given by the standard, compressed sensing channel estimation can exploit this sparsity to improve the quality of the channel estimate. In this paper, we analyze and compare the hardware complexity and denoising performance of three greedy algorithms for the 3GPP LTE system. The complexity/performance trade-off is analyzed using parameterized designs with varying configurations. One configuration of each algorithm is fabricated in a 180nm process and measured.

Evaluation and FPGA Implementation of Sparse Linear Solvers for Video Processing Applications

Sparse linear systems are commonly used in video processing applications, such as edge-aware filtering or video retargeting. Due to the 2-D nature of images, the involved problem sizes are large and thus solving such systems is computationally challenging. In this paper, we address sparse linear solvers for real-time video applications. We investigate several solver techniques, discuss hardware trade-offs, and provide field-programmable gate array (FPGA) architectures and implementation results of a Cholesky direct solver and of an iterative BiCGSTAB solver. The FPGA implementations solve 32 k × 32 k matrices at up to 50 f/s and outperform software implementations by at least one order of magnitude.

Disparity-Aware Stereo 3D Production Tools

Stereoscopic 3D (S3D) has reached wide levels of adoption in consumer and professional markets. However, production of high quality S3D content is still a difficult and expensive art. Various S3D production tools and systems have been released recently to assist high quality content creation. This paper presents a number of such algorithms, tools and systems developed at Disney Research Zurich, which all make use of disparity-aware processing.

Analysis and VLSI Implementation of EWA Rendering for Real-Time HD Video Applications

Nonlinear image warping or image resampling is a necessary step in many current and upcoming video applications, such as video retargeting, stereoscopic 3-D mapping, and multiview synthesis. The challenges for real-time resampling include not only image quality but also available energy and computational power of the employed device. In this paper, we employ an elliptical-weighted average (EWA) rendering approach to 2-D image resampling. We extend the classical EWA framework for increased visual quality and provide a very large scale integration architecture for efficient view rendering. The resulting architecture is able to render high-quality video sequences in real time targeted for low-power applications in end-user display devices.

Algorithm and VLSI architecture for real-time 1080p60 video retargeting

Aspect ratio retargeting for streaming video has actively been researched in the past years. While the mobile market with its huge diversity of screen formats is one of the most promising application areas, no existing algorithm is efficient enough to be embedded in such devices. In this work, we devise an efficient video retargeting algorithm by following an algorithm-architecture co-design approach and we present the first FPGA implementation that is able to retarget full HD 1080p video at up to 60 frames per second. We furthermore show that our algorithm can be implemented on embedded processors at interactive framerates. Our hardware architecture only requires a modest amount of hardware resources, and is portable to a dedicated ASIC for the use in consumer electronic devices such as displays or mobile phones.

Simulation and emulation of MIMO wireless baseband transceivers

The development of state-of-the-art wireless communication transceivers in semiconductor technology is a challenging process due to complexity and stringent requirements of modern communication standards such as IEEE 802.11n. This tutorial paper describes a complete design, verification, and performance characterization methodology that is tailored to the needs of the development of state-of-the-art wireless baseband transceivers for both research and industrial products. Compared to the methods widely used for the development of communication research testbeds, the described design flow focuses on the evolution of a given system specification to a final ASIC implementation through multiple design representations. The corresponding verification and characterization environment supports rapid floating-point and fixed-point performance characterization and ensures consistency across the entire design process and across all design representations. This framework has been successfully employed for the development and verification of an industrial-grade, fully standard compliant, 4-stream IEEE 802.11n MIMO-OFDM baseband transceiver.