Javier Resano

A hybrid prefetch scheduling heuristic to minimize at run-time the reconfiguration overhead of dynamically reconfigurable hardware [multimedia applications]

Due to the emergence of highly dynamic multimedia applications there is a need for flexible platforms and runtime scheduling support for embedded systems. Dynamic reconfigurable hardware (DRHW) is a promising candidate to provide this flexibility but, currently, not sufficient run-time scheduling support to deal with the run-time reconfigurations exists. Moreover, executing at run-time a complex scheduling heuristic to provide this support may generate an excessive run-time penalty. Hence, we have developed a hybrid design/run-time prefetch heuristic that schedules the reconfigurations at run-time, but carries out the scheduling computations at design-time by carefully identifying a set of near-optimal schedules that can be selected at run-time. This approach provides run-time flexibility with a negligible penalty.

A reconfigurable manager for dynamically reconfigurable hardware

Dynamic reconfiguration has been a technology solution in search of the right problem to solve. Effective use of the technology requires new programming and task management models. This article describes an approach to dynamic reconfiguration that reduces reconfiguration latency to the point where dynamic multimedia applications can now exploit such platforms.

The Promise of Reconfigurable Computing for Hyperspectral Imaging Onboard Systems: A Review and Trends

Hyperspectral imaging is an important technique in remote sensing which is characterized by high spectral resolutions. With the advent of new hyperspectral remote sensing missions and their increased temporal resolutions, the availability and dimensionality of hyperspectral data is continuously increasing. This demands fast processing solutions that can be used to compress and/or interpret hyperspectral data onboard spacecraft imaging platforms in order to reduce downlink connection requirements and perform a more efficient exploitation of hyperspectral data sets in various applications. Over the last few years, reconfigurable hardware solutions such as field-programmable gate arrays (FPGAs) have been consolidated as the standard choice for onboard remote sensing processing due to their smaller size, weight, and power consumption when compared with other high-performance computing systems, as well as to the availability of more FPGAs with increased tolerance to ionizing radiation in space. Although there have been many literature sources on the use of FPGAs in remote sensing in general and in hyperspectral remote sensing in particular, there is no specific reference discussing the state-of-the-art and future trends of applying this flexible and dynamic technology to such missions. In this work, a necessary first step in this direction is taken by providing an extensive review and discussion of the (current and future) capabilities of reconfigurable hardware and FPGAs in the context of hyperspectral remote sensing missions. The review covers both technological aspects of FPGA hardware and implementation issues, providing two specific case studies in which FPGAs are successfully used to improve the compression and interpretation (through spectral unmixing concepts) of remotely sensed hyperspectral data. Based on the two considered case studies, we also highlight the major challenges to be addressed in the near future in this emerging and fast growing research area.

FPGA Implementation of the N-FINDR Algorithm for Remotely Sensed Hyperspectral Image Analysis

Hyperspectral remote sensing attempts to identify features in the surface of the Earth using sensors that generally provide large amounts of data. The data are usually collected by a satellite or an airborne instrument and sent to a ground station that processes it. The main bottleneck of this approach is the (often reduced) bandwidth connection between the satellite and the station, which drastically limits the information that can be sent and processed in real time. A possible way to overcome this problem is to include onboard computing resources able to preprocess the data, reducing its size by orders of magnitude. Reconfigurable field-programmable gate arrays (FPGAs) are a promising platform that allows hardware/software codesign and the potential to provide powerful onboard computing capability and flexibility at the same time. Since FPGAs can implement custom hardware solutions, they can reach very high performance levels. Moreover, using run-time reconfiguration, the functionality of the FPGA can be updated at run time as many times as needed to perform different computations. Hence, the FPGA can be reused for several applications reducing the number of computing resources needed. One of the most popular and widely used techniques for analyzing hyperspectral data is linear spectral unmixing, which relies on the identification of pure spectral signatures via a so-called endmember extraction algorithm. In this paper, we present the first FPGA design for N-FINDR, a widely used endmember extraction algorithm in the literature. Our system includes a direct memory access module and implements a prefetching technique to hide the latency of the input/output communications. The proposed method has been implemented on a Virtex-4 XC4VFX60 FPGA (a model that is similar to radiation-hardened FPGAs certified for space operation) and tested using real hyperspectral data collected by NASAs Earth Observing-1 Hyperion (a satellite instrument) and the Airborne Visible Infra-Red Imaging Spectrometer over the Cuprite mining district in Nevada and the Jasper Ridge Biological Preserve in California. Experimental results demonstrate that our hardware version of the N-FINDR algorithm can significantly outperform an equivalent software version and is able to provide accurate results in near real time, which makes our reconfigurable system appealing for onboard hyperspectral data processing.

FPGA implementation of the pixel purity index algorithm for remotely sensed hyperspectral image analysis

Hyperspectral imaging is a new emerging technology in remote sensing which generates hundreds of images, at different wavelength channels, for the same area on the surface of the Earth. Over the last years, many algorithms have been developed with the purpose of finding endmembers, assumed to be pure spectral signatures in remotely sensed hyperspectral data sets. One of the most popular techniques has been the pixel purity index (PPI). This algorithm is very time-consuming. The reconfigurability, compact size, and high computational power of Field programmable gate arrays (FPGAs) make them particularly attractive for exploitation in remote sensing applications with (near) real-time requirements. In this paper, we present an FPGA design for implementation of the PPI algorithm. Our systolic array design includes a DMA and implements a prefetching technique to reduce the penalties due to the I/O communications. We have also included a hardware module for random number generation. The proposed method has been tested using real hyperspectral data collected by NASAs Airborne Visible Infrared Imaging Spectrometer over the Cuprite mining district in Nevada. Experimental results reveal that the proposed hardware system is easily scalable and able to provide accurate results with compact size in (near) real-time, which make our reconfigurable system appealing for on-board hyperspectral data processing.

Use of FPGA or GPU-based architectures for remotely sensed hyperspectral image processing

Hyperspectral imaging is a growing area in remote sensing in which an imaging spectrometer collects hundreds of images (at different wavelength channels) for the same area on the surface of the Earth. Hyperspectral images are extremely high-dimensional, and require advanced on-board processing algorithms able to satisfy near real-time constraints in applications such as wildland fire monitoring, mapping of oil spills and chemical contamination, etc. One of the most widely used techniques for analyzing hyperspectral images is spectral unmixing, which allows for sub-pixel data characterization. This is particularly important since the available spatial resolution in hyperspectral images is typically of several meters, and therefore it is reasonable to assume that several spectrally pure substances (called endmembers in hyperspectral imaging terminology) can be found within each imaged pixel. In this paper we explore the role of hardware accelerators in hyperspectral remote sensing missions and further inter-compare two types of solutions: field programmable gate arrays (FPGAs) and graphics processing units (GPUs). A full spectral unmixing chain is implemented and tested in this work, using both types of accelerators, in the context of a real hyperspectral mapping application using hyperspectral data collected by NASAs Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS). The paper provides a thoughtful perspective on the potential and emerging challenges of applying these types of accelerators in hyperspectral remote sensing missions, indicating that the reconfigurability of FPGA systems (on the one hand) and the low cost of GPU systems (on the other) open many innovative perspectives toward fast on-board and on-the-ground processing of remotely sensed hyperspectral images.

FPGA Implementation of Abundance Estimation for Spectral Unmixing of Hyperspectral Data Using the Image Space Reconstruction Algorithm

One of the most popular and widely used techniques for analyzing remotely sensed hyperspectral data is spectral unmixing, which relies on two stages: (i) identification of pure spectral signatures (endmembers) in the data, and (ii) estimation of the abundance of each endmember in each (possibly mixed) pixel. Due to the high dimensionality of the hyperspectral data, spectral unmixing is a very time-consuming task. With recent advances in reconfigurable computing, especially using field programmable gate arrays (FPGAs), hyperspectral image processing algorithms can now be accelerated for on-board exploitation using compact hardware components with small size and cost. Although in previous work several efforts have been directed towards FPGA implementation of endmember extraction algorithms, the abundance estimation step has received comparatively much less attention. In this work, we develop a parallel FPGA-based design of the image space reconstruction algorithm (ISRA), a technique for solving linear inverse problems with positive constraints that has been used to estimate the abundance of each endmember in each pixel of a hyperspectral image. It is an iterative algorithm that guarantees convergence (after a certain number of iterations) and positive values in the results of the abundances (an important consideration in unmixing applications). Our system includes a direct memory access (DMA) module and implements a pre-fetching technique to hide the latency of the input/output communications. The method has been implemented on a Virtex-4 XC4VFX60 FPGA (a model that is similar to radiation-hardened FPGAs certified for space operation) and tested using real hyperspectral data sets collected by the Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the Cuprite mining district in Nevada and the Jasper Ridge Biological Preserve in California. Experimental results demonstrate that our hardware version can significantly outperform an equivalent software version, thus being able to provide abundance estimation results in near real-time, which makes our reconfigurable system appealing for on-board hyperspectral data processing.

Specific scheduling support to minimize the reconfiguration overhead of dynamically reconfigurable hardware

Dynamically Reconfigurable Hardware (DRHW) platforms present both flexibility and high performance. Hence, they can tackle the demanding requirements of current dynamic multimedia applications, especially for embedded systems where it is not affordable to include specific HW support for all the applications. However, DRHW reconfiguration latency represents a major drawback that can make the use of DRHW resources inefficient for highly dynamic applications. To alleviate this problem, we have developed a set of techniques that provide specific support for DRHW devices and we have integrated them into an existing multiprocessor scheduling environment. In our experiments, with actual multimedia applications, we have reduced the original overhead due to the reconfiguration latency by at least 93%.

A Hardware Implementation of a Run-Time Scheduler for Reconfigurable Systems

New generation embedded systems demand high performance, efficiency, and flexibility. Reconfigurable hardware can provide all these features. However, the costly reconfiguration process and the lack of management support have prevented a broader use of these resources. To solve these issues we have developed a scheduler that deals with task-graphs at run-time, steering its execution in the reconfigurable resources while carrying out both prefetch and replacement techniques that cooperate to hide most of the reconfiguration delays. In our scheduling environment, task-graphs are analyzed at design-time to extract useful information. This information is used at run-time to obtain near-optimal schedules, escaping from local-optimum decisions, while only carrying out simple computations. Moreover, we have developed a hardware implementation of the scheduler that applies all the optimization techniques while introducing a delay of only a few clock cycles. In the experiments our scheduler clearly outperforms conventional run-time schedulers based on as-soon-as-possible techniques. In addition, our replacement policy, specially designed for reconfigurable systems, achieves almost optimal results both regarding reuse and performance.

Efficiently scheduling runtime reconfigurations

Due to the emergence of portable devices that must run complex dynamic applications there is a need for flexible platforms for embedded systems. Runtime reconfigurable hardware can provide this flexibility but the reconfiguration latency can significantly decrease the performance. When dealing with task graphs, runtime support that schedules the reconfigurations in advance can drastically reduce this overhead. However, executing complex scheduling heuristics at runtime may generate an excessive penalty. Hence, we have developed a hybrid design-time/runtime reconfiguration scheduling heuristic that generates its final schedule at runtime but carries out most computations at design-time. We have tested our approach in a PowerPC 405 processor embedded on a FPGA demonstrating that it generates a very small runtime penalty while providing almost as good schedules as a full runtime approach.