Frank Schumacher

Optimized hardware architecture of a smart camera with novel cyclic image line storage structures for morphological raster scan image processing

In this work we present an FPGA (Field Programmable Gate Array) based smart camera framework with a multiport cyclic image line storage structure and an optimized architecture for morphological raster scan image processing. The signal processing architecture is able to reuse intermediate results of 2D morphological operations, can handle different shapes of structuring elements and uses only a minimum amount of FPGA logic and block RAM resources for intermediate image line storage. This combination allows the output of one processed pixel per clock cycle and hence, build the fundament for a system architecture, which provides high frame rates with low system delays.

Matching cost computation algorithm and high speed FPGA architecture for high quality real-time Semi Global Matching stereo vision for road scenes

Stereo correspondence and the generation of the disparity map, which encodes the depth of objects, is one of the most challenging and important tasks for camera based environment perception systems. Thus, it is indispensable for autonomous driving vehicles and transportation devices to detect other cars or for the classification of obstacles. To enable this, relatively large real world images must be processed at high data rates. At the moment, Semi Global Matching (SGM) is the most promising approach for the stereo matching of real world images at sufficient quality and the capability of high data rates. Real-time SGM implementations on small image sizes have been reported, however, current stereo camera image sizes pose still high computational complexity and memory demand for SGM. This paper describes a new method for the efficient computation of stereo matching costs to reduce the complexity and the high memory demand for cost volume and cost aggregation buffering. Using the proposed complexity reduction, we present modules and concepts for full parallel FPGA implementations of the cost volume creation, SGM aggregation and disparity selection. We evaluate the presented algorithm using the KITTI stereo vision benchmark and achieve, besides competitive quality results, a data throughput for the cost calculation of 199 frames per second (fps) for an image size of 1242 × 375 with a disparity range of D = 160 and tremendously reduced memory requirements.

A real time video processing framework for hardware realization of neighborhood operations with FPGAs

In this work we present a real time video processing framework, which can handle high data throughput rates. Contrary to common digital hardware realizations which use several image line long shift register pipelines for direct calculation of 2D neighborhood operations, we suggest an efficient cyclic image line storage structure by using dual port block RAM buffers, which are available in recent FPGAs. Therefore, our approach does not occupy a huge amount of valuable logic resources in the FPGA for shift registers based data storage and achieves a high data throughput by parallel video data processing paths. With this memory structure we realize — already principally proposed in a previous work — a new hardware architecture of the basic morphological image processing operations erosion and dilation as building blocks. With these building blocks, which are hardware occupation and maximum clock frequency efficient, we also implemented the combined morphological operations opening and closing, which are commonly used for image enhancement like noise reduction and object contour smoothing.

Modeling and code generation of recursive algorithms with extended UML Activity Diagrams

While most current graphical modeling languages for specifying digital signal processing algorithms provide a rich set of loop techniques and execution semantics based on IP-libraries, they lack in general of the flexibility of model based development frameworks. Also, the potential of recursive and repetitive graphical descriptions is mostly not utilized. We present a novel graphical notation to describe digital signal processing systems which enables the general specification of repetitive and recursive algorithms with hierarchical and parallel behavior diagrams. Our approach closes also the gap to flexible model-driven approaches by using extended UML Activity Diagrams, which enable the use of model based techniques including abstract descriptions, model-to-model-transformations and code generation for rapid prototyping of hardware and software systems.

Critical path minimized raster scan hardware architecture for computation of the Generalized Hough Transform

The Generalized Hough Transform (GHT) is a well known image processing transform to find arbitrary shapes in images. We propose a new raster scan FPGA and VSLI hardware architecture, performing the GHT of a binary template shape with an input image. The architecture has a minimized critical path by the reuse of partial results and the utilization of a flat adder structure. A synchronous pixel pipeline for input image row buffering enables parallel read and write access of the partial results. By this, the architectures critical path and hence the maximum clock frequency is independent of the template size, content and number of valid pixels in the template. The architecture was implemented on a Xilinx Virtex 5 FPGA device. The design reaches a pixel clock of more than 580 MHz at an exemplary image size of 512×512 pixels and a template size of 64×64 pixels.

Shape independent VLSI-architecture design approach for 2D morphological operations with non-flat structuring elements

In this work we present a structuring element shape independent Very Large Scale Integration (VLSI) design approach for 2D morphological operations, which is capable to handle flat and non-flat structuring elements. Contrary to other architectural concepts which are commonly based on structuring element areal decomposition and a sequential calculation approach, our approach requires no intermediate image data storage over several image lines. In addition, the amount of comparators, registers and for non-flat structuring elements adders / subtractors is reduced significantly by reusing results on so called orthogonal shift levels. This leads to hardware efficient and fast VLSI realizations of 2D morphological operations.

Contour chain-coding and topological hierarchy analysis in a 2×3 window single-pass raster scan

This paper presents a fast and effective method for generating chain-encoded representations of arbitrary contours of connected regions in a binary image. Furthermore, an optional method is presented allowing the additional concurrent extraction of the topological hierarchy tree within the image. Both parts of the algorithm are executed by means of a single raster scan with a 2×3 neighborhood convolution window. This not only allows the reduction of the pixel buffer to a single image line, it furthermore enables fast neighborhood evaluation by means of a moderately sized look-up-table with 64 entries, to be directly addressed by the six pixel in window focus. With these main characteristics, the algorithm is well suited for efficient parallel processing implementations in streaming applications.