Miguel Arias-Estrada

An FPGA Correlation-Edge Distance approach for disparity map

This paper describes an FPGA Correlation-Edge Distance approach for real time disparity map generation in stereo-vision. The proposed method calculates the disparity map for the input and disparity map for Edge Distance images of a stereopair. In both cases the approximation algorithm of disparity map SAD (Sum of Absolute Differences) is used. The final disparity map is determined from the previously generated maps, considering a homogeneity parameter defined for each point in the scene. Due to low complexity when implementing stereo-vision algorithms in FPGA devices, the proposed method was implemented in a Cyclone II EP2C35F672C6 FPGA assembled in an Altera DE2 breadboard. The developed module can process stereo-pairs of 1280×1024 pixel resolution at a rate of 75 frames/s and produces 8-bit dense disparity maps within a range of disparities up to 63 pixels. The presented architecture provides a significant improvement in regions with uniformed texture over correlation based stereo-vision algorithms in the reported literature and an accelerated processing rate.

An FPGA Stereo Matching Processor Based on the Sum of Hamming Distances

Stereo matching is a useful algorithm to infer depth information from two or more of images and has uses in mobile robotics, three-dimensional building mapping and three-dimensional reconstruction of objects. In area-based algorithms, the similarity between one pixel of an image key frame and one pixel of another image is measured using a correlation index computed on neighbors of these pixels correlation windows. In order to preserve edges, the use of small correlation windows is necessary while for homogeneous areas, large windows are required. In addition, to improve the execution time, stereo matching algorithms often are implemented in dedicated hardware such as FPGA or GPU devices. In this article, we present an FPGA stereo matching processor based on the Sum of Hamming Distances SHD. We propose a grayscale-based similarity criterion, which allows separating the objects and background from the correlation window. By using the similarity criterion, it is possible to improve the performance of any grayscale-based correlation coefficient and reach high performance for homogeneous areas and edges. The developed FPGA architecture reaches high performance compared to other real-time stereo matching algorithms, upi?źto 10i?ź% more accuracy and enables to increase the processing speed near to 20 megapixels per second.

A Fuzzy Logic Approach for Stereo Matching Suited for Real-Time Processing

In this paper, a novel method that uses both area and feature based information as similarity measures for stereo matching is proposed. Area-based information is suited for non-homogeneous regions while feature information helps in homogeneous areas. In order to define a conjugate pair, a fuzzy logic approach that combines the similarity information is used. The proposed method preserves discontinuities while reducing matching errors in homogeneous regions. This proposal is suited for real-time processing using dedicated hardware. We demonstrate and discuss performance using synthetic stereo pairs.

Parallel processor for 3D recovery from optical flow

In recent years, 3D recovery from motion has received a major effort in computer vision systems. The main problem lies in the number of operations and memory accesses to be performed by the majority of the existing techniques when translated to hardware or software implementations. This work proposes a parallel processor for 3D recovery from optical flow. Its main feature is the maximum reuse of data and the low number of clock cycles to calculate the optical flow, along with the precision with which 3D recovery is achieved. The results of such recovery, as well as those from processor synthesis are presented.

Parallel Processor for 3D Recovery from Optical Flow

In recent years, 3D recovery from motion has received a major effort in computer vision systems. The main problem lies in the number of operations and memory accesses to be performed by the majority of the existing techniques when translated to hardware or software implementations. This work proposes a parallel processor for 3D recovery from optical flow. Its main feature is the maximum reuse of data and the low number of clock cycles to calculate the optical flow, along with the precision with which 3D recovery is achieved. The results of such recovery, as well as those from processor synthesis are presented.

Robust feature extraction algorithm suitable for real-time embedded applications

Smart cameras integrate processing close to the image sensor, so they can deliver high-level information to a host computer or high-level decision process. One of the most common processing is the visual features extraction since many vision-based use-cases are based on such algorithm. Unfortunately, in most of cases, features detection algorithms are not robust or do not reach real-time processing. Based on these limitations, a feature detection algorithm that is robust enough to deliver robust features under any type of indoor/outdoor scenarios is proposed. This was achieved by applying a non-textured corner filter combined to a subpixel refinement. Furthermore, an FPGA architecture is proposed. This architecture allows compact system design, real-time processing for Full HD images (it can process up to 44 frames/91.238.400 pixels per second for Full HD images), and high efficiency for smart camera implementations (similar hardware resources than previous formulations without subpixel refinement and without non-textured corner filter). For accuracy/robustness, experimental results for several real-world scenes are encouraging and show the feasibility of our algorithmic approach.

Dense mapping for monocular-SLAM

Simultaneous Localization and Mapping (SLAM) is the problem of constructing a 3D map while simultaneously keeping track of an agent location within the map. In recent years, work has focused in systems that use a single camera as the only sensing mechanism (monocular-SLAM). 3D reconstruction (map) by monocular-SLAM systems is a point cloud where all points preserve high accuracy and can deliver visual environmental information. However, the maximum number of points in the cloud is limited by the tracked features, this is named “sparse cloud problem”. In this work, we propose a new SLAM framework that is robust enough for indoor/outdoor SLAM applications, and at the same time increases the 3D map density. The point cloud density is increased by applying a new feature-tracking/dense-tracking algorithm in the SLAM formulation. In order to achieve real-time processing, the algorithm is formulated to facilitate a parallel FPGA implementation. Preliminary results show that it is possible to obtain dense mapping (superior to previous work) and accurate camera pose estimation (localization) under several real-world conditions.

Towards a smart camera for monocular SLAM

In recent years the interest on monocular-SLAM (Simultaneous Localization and Mapping) has increased, this because nowadays it is possible to find inexpensive, small and light commercial cameras and they provide visual environmental information that can be exploited to create 3D maps and camera pose in an unknown environment. A smart camera that could deliver monocular-SLAM is highly desirable, since it can be the basis of several robotics/drone applications. In this article, we present a new SLAM framework that is robust enough for indoor/outdoor SLAM applications, and at the same time is parallelizable in the context of FPGA architecture design. We introduce new feature-extraction/feature-matching algorithms, suitable for FPGA implementation. We propose an FPGA based sensor-processor architecture where most of the visual processing is carried out in a parallel architecture, and the 3D map construction and camera pose estimation in the processor of a SoC FPGA. An FPGA architecture is lay down and hardware/software partition is discussed. We show that the proposed sensor-processor can deliver high performance under several indoor/outdoor scenarios.

A Low-Level Image Processing Algorithms Accelerator Platform

This work presents a reconfigurable computing architecture targeting a Virtex-II device. In this approach a systolic array for low-level image processing is considered. The architecture is customizable providing the possibility of performing window operations with masks of variable size and every processing element in the array can be configured according to a control word. The architecture comprises a scheme to reduce the number of accesses to data memory and router elements to handle data movement among different structures inside the same architecture, these components add the possibility of chaining interconnection of multiple processing blocks. In order to turn the architecture into a real platform, support has been provided to the motion estimation algorithm which presents higher complexity. In this modality the 2D array operates with a double ALU that allows searching multiple macro-blocks in parallel. Results using 640 times 480 gray level images show that a peak performance of 9 GOPS can be achieved.

Deep Learning Pulsed-based Convolutional Neuroprocessor Architecture on FPGAs

The biological human brain works with spiking neural networks which computational complexity is simple, compensated by the high density connection between neurons. However, most of our research in artificial neural networks is based on simplified models that need many real values processing elements of complex computing which require too much silicon space, energy and slow learning convergence. Therefore, in this brief article, it is presented a proposal for a fully digital architecture on FPGAs optimized for a highly dense intersynaptic connection on an event-based quantized Sigma-Delta pulse coding for deep Convolutional Neural Networks. This article presents early results of an approach of neuromorphic hardware design for information or pixel luminosity changes coded in time, using Sigma-Delta modulation, the design of a Pulsed Arithmetic-Logic Unit for bitstream operations with quantized weights reducing memory, from 32 bits in floating point representation, down to 1 bit. This 32x memory reduction and binary operation cells in a systolic architectures makes the integration of deep learning models feasible for embedded design like smart cameras using FPGAs.