Benny Thörnberg

Exploration of Tasks Partitioning between Hardware Software and Locality for a Wireless Camera Based Vision Sensor Node

In this paper we have explored different possibilities for partitioning the tasks between hardware, software and locality for the implementation of the vision sensor node, used in wireless vision sensor network. Wireless vision sensor network is an emerging field which combines image sensor, on board computation and communication links. Compared to the traditional wireless sensor networks which operate on one dimensional data, wireless vision sensor networks operate on two dimensional data which requires higher processing power and communication bandwidth. The research focus within the field of wireless vision sensor networks have been on two different assumptions involving either sending raw data to the central base station without local processing or conducting all processing locally at the sensor node and transmitting only the final results. Our research work focus on determining an optimal point of hardware/software partitioning as well as partitioning between local and central processing, based on minimum energy consumption for vision processing operation. The lifetime of the vision sensor node is predicted by evaluating the energy requirement of the embedded platform with a combination of FPGA and micro controller for the implementation of the vision sensor node. Our results show that sending compressed images after pixel based tasks will result in a longer battery life time with reasonable hardware cost for the vision sensor node.

Road Surface Status Classification Using Spectral Analysis of NIR Camera Images

There is a need for an automated road status classification system considering the vast number of weather-related accidents that occur every winter. Previous research has shown that it is possible to detect hazardous road conditions, including, for example, icy pavements, using single point infrared illumination and infrared detectors. In this paper, we extend this research into camera surveillance of a road section allowing for classification of area segments of weather-related road surface conditions such as wet, snow covered, or icy. Infrared images have been obtained using an infrared camera equipped with a set of optical wavelength filters. The images have primarily been used to develop multivariate data models and also for the classification of road conditions in each pixel. This system is a vast improvement on existing single spot road status classification systems. The resulting imaging system can reliably distinguish between dry, wet, icy, or snow covered sections on road surfaces.

Bit-Width Constrained Memory Hierarchy Optimization for Real-Time Video Systems

The great variety of pixel dynamics of real-time video-processing systems (RTVPS), ranging from color, grayscale, or binary pixels, means that a careful design and specification of bit widths is required. It is obvious that the bit-width specification will affect the total memory storage requirement. However, what is not so obvious is that the bit-width specification will also affect the design of the memory hierarchy, an impact similar for both hardware and software implementations. We have developed an integer-nonlinear-program formulation for the optimization of the memory hierarchy of RTVPS. An active surveillance video camera is introduced as a test case. We demonstrate how the optimization model can reduce the on-chip memory storage by 61% compared to a nonoptimal memory hierarchy

Address generation for FPGA RAMS for efficient implementation of real-time video processing systems

FPGA offers the potential of being a reliable, and high-performance reconfigurable platform for the implementation of real-time video processing systems. To utilize the full processing power of FPGA for video processing applications, optimization of memory accesses and the implementation of memory architecture are important issues. This paper presents two approaches, base pointer approach and distributed pointer approach, to implement accesses to on-chip FPGA Block RAMs. A comparison of the experimental results obtained using the two approaches on realistic image processing systems design cases is presented. The results show that compared to the base pointer approach the distributed pointer approach increases the potential processing power of FPGA, as a reconfigurable platform for video processing systems.

Hardware Architecture for Real-time Computation of Image Component Feature Descriptors on a FPGA

This paper describes a hardware architecture for real-time image component labeling and the computation of image component feature descriptors. These descriptors are object related properties used to describe each image component. Embedded machine vision systems demand a robust performance and power efficiency as well as minimum area utilization, depending on the deployed application. In the proposed architecture, the hardware modules for component labeling and feature calculation run in parallel. A CMOS image sensor (MT9V032), operating at a maximum clock frequency of 27 MHz, was used to capture the images. The architecture was synthesized and implemented on a Xilinx Spartan-6 FPGA. The developed architecture is capable of processing 390 video frames per second of size 640 × 480 pixels. Dynamic power consumption is 13 mW at 86 frames per second.

Polyhedral space generation and memory estimation from interface and memory models of real-time video systems

We present a tool and a methodology for estimating the memory storage requirement for synchronous real-time video processing systems. Typically, a designer will use the feedback information from this estimation to select the most optimal execution order for software processors or space to time mapping for hardware. We propose to start from a conceptual interface and memory model that captures memory usage and data transfers. This high-level modeling is provided as an extension library of SystemC called IMEM. A common polyhedral iteration space is generated from the model, where polytopes are placed using a new placement algorithm based on simple heuristics. This algorithm will ensure maximum freedom of selecting executing order as all negative dependencies are removed to the length of zero. A demonstration is given regarding how the polytopes and dependency vectors can then be used as input to a memory storage estimation tool called STOREQ.

High-performance FPGA based camera architecture for range imaging

Range imaging is often used in classification of objects in process industry. The speed of inspection needs to be high, so it does not become the bottleneck in the process. This paper presents an FPGA based architecture for range imaging. Using centre of gravity it calculates the range positions from 2D images. The results show that the proposed architecture can process range values with a performance up to 150 Msamples per second. Thus, using cheep standard technology we can achieve up to 3 times higher performance than expensive state-of-the-art high performance smart-cameras.

Strategic Proactive Obsolescence Management Model

When there is a demand larger than the corresponding number of components in stock, obsolescence problems may occur for those systems with a life cycle longer than that of one or more of their components, such as automotive, avionics, military application, etc. This paper discusses the electronic component obsolescence problem and presents a formal mathematical strategic proactive obsolescence management model for long life-cycle systems. The model presented in this paper utilizes redesign and last-time-buy (LTB) as two management methods. The LTB cost is estimated by unit cost, demand quantities, buffer, discount rate, and holding cost. Redesign cost is associated with component type and quantities. This model can estimate the minimum management costs for a system with different architectures. It consists of two parts. The first is to generate a graph, which is in the form of an obsolescence management diagram. A segments table containing the data of this diagram is calculated and prepared for optimization at a second step. The second part is to find the minimum cost for system obsolescence management. Mixed integer linear programming is used to calculate the minimum management cost and schedule. The model is open sourced allowing other research groups to freely download and modify it. A display and control system case study is shown to apply this model practically. A reactive manner is presented as a comparison. The result of the strategic proactive management model shows significant cost avoidance compared to the reactive manner.

Power-aware automatic constraint generation for FPGA based real-time video processing systems

The introduction of embedded DSP blocks and embedded memory has made FPGAs an attractive architecture for implementation of real-time video processing systems. The big bottle neck of the FPGA compared to other programmable architectures is the complex programming model. This paper presents an automatic generation of placement and routing constraints for FPGA implementation of real-time video processing systems as one step to automate the programming model. The constraint generator targets lower power consumption, better resource utilization and reduced development time. Results show that a 28% reduction in dynamic power can be achieved using the proposed approach over traditional logic to memory mapping.

C++ based System Synthesis of Real-Time Video Processing Systems targeting FPGA Implementation

Implementing real-time video processing systems put high requirements on computation and memory performance. FPGAs have proven to be effective implementation architecture for these systems. However, the hardware based design flow for FPGAs make the implementation task complex. The system synthesis tool presented in this paper reduces this design complexity. The synthesis is done from a SystemC based coarse grain dataflow graph that captures the video processing system. The data flow graph is optimized and mapped onto an FPGA. The results from real-life video processing systems clearly show that the presented tool produces effective implementations.