Nikos Sgouros

Enhanced reconstruction of three-dimensional shape and texture from integral photography images

A method for the reconstruction of 3D shape and texture from integral photography (IP) images is presented. Sharing the same principles with stereoscopic-based object reconstruction, it offers increased robustness to noise and occlusions due to the unique characteristics of IP images. A coarse-to-fine approach is used, employing what we believe to be a novel grid refinement step in order to increase the quality of the reconstructed objects. The proposed methods properties include configurable depth accuracy and direct and seamless triangulation. We evaluate our method using synthetic data from a computer-simulated IP setup as well as real data from a simple yet effective digital IP setup. Experiments show reconstructed objects of high-quality indicating that IP can be a competitive modality for 3D object reconstruction.

A real-time FPGA architecture for 3D reconstruction from integral images

In this paper, we present a hardware architecture for real-time three-dimensional (3D) surface model reconstruction from Integral Images (InIms). The proposed parallel digital system realizes a number of computational-heavy calculations in order to achieve real-time operation. The processing elements are deployed in a systolic architecture and operate on multiple image areas simultaneously. Moreover, memory organization allows random access to image data and copes with the increased processing throughput of the system. Operating results reveal that the proposed architecture is able to process 3D data at a real-time rate. The proposed system can handle large sized InIms in real time and outputs 3D scenes of enhanced depth and detailed texture, which apply to emerging 3D applications.

Hardware implementation of a disparity estimation scheme for real-time compression in 3D imaging applications

This paper presents a novel hardware implementation of a disparity estimation scheme targeted to real-time Integral Photography (IP) image and video sequence compression. The software developed for IP image compression achieves high quality ratios over classic methodologies by exploiting the inherent redundancy that is present in IP images. However, there are certain time constraints to the software approach that must be confronted in order to address real-time applications. Our main effort is to achieve real-time performance by implementing in hardware the most time-consuming parts of the compression algorithm. The proposed novel digital architecture features minimized memory read operations and extensive simultaneous processing, while taking into concern the memory and data bandwidth limitations of a single FPGA implementation. Our results demonstrate that the implemented hardware system can successfully process high resolution IP video sequences in real-time, addressing a vast range of applications, from mobile systems to demanding desktop displays.

Robust integral image rectification framework using perspective transformation supported by statistical line segment clustering

In most integral image analysis and processing tasks, accurate knowledge of the internal image structure is required. In this paper we present a robust framework for the accurate rectification of perspectively distorted integral images based on multiple line segment detection. The use of multiple line segments increases the overall fault tolerance of our framework providing strong statistical support for the rectification process. The proposed framework is used for the automatic rectification, metric correction, and rotation of distorted integral images. The performance of our framework is assessed over a number of integral images with varying scene complexity and noise levels.

Hardware Acceleration for 3D Image Reconstruction

This work presents a hardware acceleration scheme for a 3D reconstruction method, targeting demanding dynamic Integral Imaging applications. The architecture exploits parallel processing and minimizes memory operations by implementing an efficient metric and an extended-access memory scheme. The employed data reutilization technique reduces overall throughput allowing the use of a single FPGA. Results reveal that the hardware system accelerates the software method by an order of magnitude and its processing rate surpasses the typical rate of a dynamic Integral Imaging acquisition system, making it suitable for robust 3D image and video reconstruction applications.

Near Real-Time 3D Reconstruction from InIm Video Stream

An efficient hardware architecture for the acceleration of an integrated 3D reconstruction method is presented, targeting demanding dynamic Integral Imaging applications. It exploits parallel processing and features minimized memory operations by implementing an extended-access memory scheme. Its reduced data throughput, thanks to optimized data utilization, makes it suitable for single FPGA device implementation. Results reveal that the hardware system outperforms the software method by an order of magnitude. Moreover, its processing rate surpasses the typical rate of a dynamic Integral Imaging acquisition system, thus making a significant step towards real-time 3D video reconstruction.

FPGA-based architecture for real-time IP video and image compression

Three-dimensional imaging applications require high resolution images that finally result in high data volumes. Due to bandwidth and storage restrictions, an efficient and robust compression scheme must be developed in order to overcome these limitations. This work presents a hardware implementation of a real-time disparity estimation scheme targeted but not limited to integral photography (IP) 3D imaging applications. The proposed system demonstrates an efficient architecture which copes with the increased bandwidth demands that 3D imaging technology requires. Moreover, the system can successfully process high resolution IP video sequences in real-time

Perspective rectification of integral images produced using arrays of circular lenses

There are many different three-dimensional (3D) techniques to capture and deliver autostereoscopic 3D content. A promising technique that provides two-dimensional parallax as well as high-quality, full-color 3D content is integral imaging (InI). Misalignments between the lens arrays (LAs) and the camera charged coupled device, however, introduce geometric distortions in the acquired image that propagate through the different image processing stages and deteriorate the 3D effect. Here, we propose a method to accurately rectify the perspective distortion of integral images (InIms) generated using circular lenses. Using an edge-linking approach, we extracted elliptically shaped contours of elemental images in the perspectively distorted InIm. To calculate the rectification matrix, we used the images of the circular points. Subsequently, we applied a triangulation scheme followed by a statistical approach to accurately estimate the grid structure of the LA. Finally, we provided experimental results over a wide range of InIms to evaluate the robustness and accuracy of the proposed method using objective metrics.

Real-time compression architecture for efficient coding in autostereoscopic displays

Integral imaging is a promising technique for delivering high-quality three-dimensional content. However, the large amounts of data produced during acquisition prohibits direct transmission of Integral Image data. A number of highly efficient compression architectures are proposed today that outperform standard two-dimensional encoding schemes. However, critical issues regarding real-time compression for quality demanding applications are a primary concern to currently existing Integral Image encoders. In this work we propose a real-time FPGA-based encoder for Integral Image and integral video content transmission. The proposed encoder is based on a highly efficient compression algorithm used in Integral Imaging applications. Real-time performance is achieved by realizing a pipelined architecture, taking into account the specific structure of an Integral Image. The required memory access operations are minimized by adopting a systolic concept of data flow through the core processing elements, further increasing the performance boost. The encoder targets, real-time, broadcast-type high-resolution Integral Image and video sequences and performs three orders of magnitude faster than the analogous software approach.

Real-time processing pipeline for 3D imaging applications

Most processing tasks in three-dimensional (3D) applications share common ground in terms of algorithm design and implementation. Furthermore, time consuming tasks like 3D reconstruction of objects from generic multiview images and compression of massive amounts of 3D image data, require enormous data processing power, and in many cases computations should be performed in real-time. In this paper we present a hardware architecture that jointly addresses both tasks. The proposed design features processing elements and memory modules that form a common compression and reconstruction datapath. Extensive pipelining minimizes throughput delays and memory access operations. Our implementation achieves real-time performance for both tasks, efficiently addressing demanding 3D imaging and video applications.