Eduardo Magdaleno

Fourier Slice Super-resolution in plenoptic cameras

Plenoptic cameras are a promising solution to increase the capabilities of current commercial cameras because they capture the four-dimensional lightfield of a scene. Processing the recorded lightfield, these cameras offer the possibility of focusing the scene after the shot or obtaining 3D information. Conventional photographs focused on determined planes can be obtained through projections of the four-dimensional lightfield onto two spatial dimensions. These photographs can be efficiently computed using the Fourier Slice technique but their resolution is usually less than 1% of the full resolution of the camera sensor. Several super-resolution methods have been recently developed to increase the spatial resolution of plenoptic cameras. In this paper we propose a new super-resolution method based on the Fourier Slice technique. We show how several existing super-resolution methods can be seen as particular cases of this approach. Besides the theoretical interest of this unified view, we also show how to obtain simultaneously spatial and depth super-resolution removing the limitations of previous approaches.

Super-Resolution in Plenoptic Cameras Using FPGAs

Plenoptic cameras are a new type of sensor that extend the possibilities of current commercial cameras allowing 3D refocusing or the capture of 3D depths. One of the limitations of plenoptic cameras is their limited spatial resolution. In this paper we describe a fast, specialized hardware implementation of a super-resolution algorithm for plenoptic cameras. The algorithm has been designed for field programmable graphic array (FPGA) devices using VHDL (very high speed integrated circuit (VHSIC) hardware description language). With this technology, we obtain an acceleration of several orders of magnitude using its extremely high-performance signal processing capability through parallelism and pipeline architecture. The system has been developed using generics of the VHDL language. This allows a very versatile and parameterizable system. The system user can easily modify parameters such as data width, number of microlenses of the plenoptic camera, their size and shape, and the super-resolution factor. The speed of the algorithm in FPGA has been successfully compared with the execution using a conventional computer for several image sizes and different 3D refocusing planes.

An Efficient Pipeline Wavefront Phase Recovery for the CAFADIS Camera for Extremely Large Telescopes

In this paper we show a fast, specialized hardware implementation of the wavefront phase recovery algorithm using the CAFADIS camera. The CAFADIS camera is a new plenoptic sensor patented by the Universidad de La Laguna (Canary Islands, Spain): international patent PCT/ES2007/000046 (WIPO publication number WO/2007/082975). It can simultaneously measure the wavefront phase and the distance to the light source in a real-time process. The pipeline algorithm is implemented using Field Programmable Gate Arrays (FPGA). These devices present architecture capable of handling the sensor output stream using a massively parallel approach and they are efficient enough to resolve several Adaptive Optics (AO) problems in Extremely Large Telescopes (ELTs) in terms of processing time requirements. The FPGA implementation of the wavefront phase recovery algorithm using the CAFADIS camera is based on the very fast computation of two dimensional fast Fourier Transforms (FFTs). Thus we have carried out a comparison between our very novel FPGA 2D-FFTa and other implementations.

A FPGA embedded web server for remote monitoring and control of smart sensors networks.

This article describes the implementation of a web server using an embedded Altera NIOS II IP core, a general purpose and configurable RISC processor which is embedded in a Cyclone FPGA. The processor uses the μCLinux operating system to support a Boa web server of dynamic pages using Common Gateway Interface (CGI). The FPGA is configured to act like the master node of a network, and also to control and monitor a network of smart sensors or instruments. In order to develop a totally functional system, the FPGA also includes an implementation of the time-triggered protocol (TTP/A). Thus, the implemented master node has two interfaces, the webserver that acts as an Internet interface and the other to control the network. This protocol is widely used to connecting smart sensors and actuators and microsystems in embedded real-time systems in different application domains, e.g., industrial, automotive, domotic, etc., although this protocol can be easily replaced by any other because of the inherent characteristics of the FPGA-based technology.

Super-Resolved Fourier-Slice Refocusing in Plenoptic Cameras

Plenoptic cameras are a new type of sensors that capture the four-dimensional lightfield of a scene. Processing the recorded lightfield, these cameras extend the capabilities of current commercial cameras offering the possibility of focusing the scene after the shot or obtaining 3D information. Conventional photographs focused on certain planes can be obtained through projections of the four-dimensional lightfield onto two spatial dimensions. These photographs can be efficiently computed using the Fourier Slice technique, but their resolution is limited since a plenoptic camera trades off spatial resolution for angular resolution. In order to remove this limitation, several super-resolution methods have been recently developed to increase the spatial resolution of plenoptic cameras. In this paper, we study the super-resolution problem in plenoptic cameras and show how to efficiently compute super-resolved photographs using the Fourier Slice technique. We also show how several existing super-resolution methods can be seen as particular cases of this approach. Experimental results are provided to show the validity of the approach and its extension to super-resolved all-in-focus image computation and 3D processing is studied.

A fast and memory-efficient Discrete Focal Stack Transform for plenoptic sensors

Plenoptic cameras are a new type of sensors that capture the four-dimensional lightfield of a scene. Processing the recorded lightfield, they extend the capabilities of current commercial cameras. Conventional cameras obtain photographs focusing at a determined depth. This photograph can be described through a projection of the four-dimensional lightfield onto two spatial dimensions. The collection of such images is the focal stack of the scene. The focal stack can be used to select an image refocused at a certain depth, to recover 3D information or to obtain all-in-focus images. There are several approaches to the computation of the focal stack. In this paper we propose a new technique to compute the focal stack by means of its frequency decomposition that can be seen as an extension of the Discrete Focal Stack Transform (DFST). This new approach decreases the computational complexity of the DFST maintaining an efficient memory use. Experimental results are provided to show the validity of the technique and its extension to 3D processing and all-in-focus image computation is also studied.

Design of Belief Propagation Based on FPGA for the Multistereo CAFADIS Camera

In this paper we describe a fast, specialized hardware implementation of the belief propagation algorithm for the CAFADIS camera, a new plenoptic sensor patented by the University of La Laguna. This camera captures the lightfield of the scene and can be used to find out at which depth each pixel is in focus. The algorithm has been designed for FPGA devices using VHDL. We propose a parallel and pipeline architecture to implement the algorithm without external memory. Although the BRAM resources of the device increase considerably, we can maintain real-time restrictions by using extremely high-performance signal processing capability through parallelism and by accessing several memories simultaneously. The quantifying results with 16 bit precision have shown that performances are really close to the original Matlab programmed algorithm.

Modal Fourier Wavefront Reconstruction Using FPGA Technology

Atmospheric turbulence introduces optical aberration into wavefronts arriving at ground-based telescopes. The wavefront reconstruction using fast Fourier transforms (FFT) and spatial filtering is computationally tractable and sufficiently accurate in large Shack-Hartmann-based adaptive optics systems (up to 10,000 actuators). A first design has been developed using a Graphical Processing Units (GPU) platform. However, an increase in telescope size requires significant computational power. For this reason other hardware technologies must be taken into account during the development of a specific processor. In this paper, an improvement on the efficiency of the above methods is proposed based on Field Programmable Gate Array (FPGA) technology. To the best of our knowledge, there are not other wavefront reconstruction implementations based on FPGA technology. The basic advantages of this technology are its flexible architecture and extremely highperformance signal processing capability which are captured through parallelism. The implemented phase recoverer is, consequently, faster than the CPU or the GPU solution. Furthermore, the implemented design incorporating this technology meets current and future adaptive optics image processing frame rate requirements.

Lightfield Recovery from Its Focal Stack

The Focal Stack Transform integrates a 4D lightfield over a set of appropriately chosen 2D planes. The result of such integration is an image focused on a determined depth in 3D space. The set of such images is the Focal Stack of the lightfield. This paper studies the existence of an inverse for this transform. Such inverse could be used to obtain a 4D lightfield from a set of images focused on several depths of the scene. In this paper, we show that this inversion cannot be obtained for a general lightfield and introduce a subset of lightfields where this inversion can be computed exactly. We examine the numerical properties of such inversion process for general lightfields and examine several regularization approaches to stabilize the transform. Experimental results are provided for focal stacks obtained from several plenoptic cameras. From a practical point of view, results show how this inversion procedure can be used to recover, compress, and denoise the original 4D lightfield.

Automated Software Acceleration in Programmable Logic for an Efficient NFFT Algorithm Implementation: A Case Study

Non-equispaced Fast Fourier transform (NFFT) is a very important algorithm in several technological and scientific areas such as synthetic aperture radar, computational photography, medical imaging, telecommunications, seismic analysis and so on. However, its computation complexity is high. In this paper, we describe an efficient NFFT implementation with a hardware coprocessor using an All-Programmable System-on-Chip (APSoC). This is a hybrid device that employs an Advanced RISC Machine (ARM) as Processing System with Programmable Logic for high-performance digital signal processing through parallelism and pipeline techniques. The algorithm has been coded in C language with pragma directives to optimize the architecture of the system. We have used the very novel Software Develop System-on-Chip (SDSoC) evelopment tool that simplifies the interface and partitioning between hardware and software. This provides shorter development cycles and iterative improvements by exploring several architectures of the global system. The computational results shows that hardware acceleration significantly outperformed the software based implementation.