Wilfried Kubinger

An Optimized Software-Based Implementation of a Census-Based Stereo Matching Algorithm

This paper presents S 3 E , a software implementation of a high-quality dense stereo matching algorithm. The algorithm is based on a Census transform with a large mask size. The strength of the system lies in the flexibility in terms of image dimensions, disparity levels, and frame rates. The program runs on standard PC hardware utilizing various SSE instructions. We describe the performance optimization techniques that had a considerably high impact on the run-time performance. Compared to a generic version of the source code, a speedup factor of 112 could be achieved. On input images of 320×240 and a disparity range of 30, S 3 E achieves 42fps on an Intel Core 2 Duo CPU running at 2GHz.

SAD-Based Stereo Matching Using FPGAs

In this chapter we present a field-programmable gate array (FPGA) based stereo matching architecture. This architecture uses the sum of absolute differences (SAD) algorithm and is targeted at automotive and robotics applications. The disparity maps are calculated using 450×375 input images and a disparity range of up to 150 pixels. We discuss two different implementation approaches for the SAD and analyze their resource usage. Furthermore, block sizes ranging from 3×3 up to 11×11 and their impact on the consumed logic elements as well as on the disparity map quality are discussed. The stereo matching architecture enables a frame rate of up to 600 fps by calculating the data in a highly parallel and pipelined fashion. This way, a software solution optimized by using Intel’s Open Source Computer Vision Library running on an Intel Pentium 4 with 3 GHz clock frequency is outperformed by a factor of 400.

Flexible hardware-based stereo matching

To enable adaptive stereo vision for hardware-based embedded stereo vision systems, we propose a novel technique for implementing a flexible block size, disparity range, and frame rate. By reusing existing resources of a static architecture, rather than dynamic reconfiguration, our technique is compatible with application specific integrated circuit (ASIC) as well as field programmable gate array (FPGA) implementations. We present the corresponding block diagrams and their implementation in our hardware-based stereo matching architecture. Furthermore, we show the impact of flexible stereo matching on the generated disparity maps for the sum of absolute differences (SADs), rank, and census transform algorithms. Finally, we discuss the resource usage and achievable performance when synthesized for an Altera Stratix II FPGA.

SciAutonics-Auburn Engineering’s Low Cost High Speed ATV for the 2005 DARPA Grand Challenge

This paper presents a summary of SciAutonics-Auburn Engineering’s efforts in the 2005 DARPA Grand Challenge. The areas discussed in detail include the team makeup and strategy, vehicle choice, software architecture, vehicle control, navigation, path planning, and obstacle detection. In particular, the advantages and complications involved in fielding a low budget all-terrain vehicle are presented. Emphasis is placed on detailing the methods used for high-speed control, customized navigation, and a novel stereo vision system. The platform chosen required a highly accurate model and a well-tuned navigation system in order to meet the demands of the Grand Challenge. Overall, the vehicle completed three out of four runs at the National Qualification Event and traveled 16 miles in the Grand Challenge before a hardware failure disabled operation. The performance in the events is described, along with a success and failure analysis.

RASCAL - an autonomous ground vehicle for desert driving in the DARPA Grand Challenge 2005

The DARPA Grand Challenge is a competition of autonomous ground vehicles in the Mojave desert, with a prize of } for the winner. This event was organized in 2004 and held annually at least until 2007, until a team wins the prize. The teams are coming from various background, but the rule that no US government funding or technology that was created with US government funding could be used for this competition, prevented some of the well established players to participate. The team SciAutonics/Auburn-Engineering continues their effort to build a system for participation in this challenge, based on the 2004 entry RASCAL. The main focus in the system design is on improvements of the design from 2004. Novel sensing modalities the team plans to use in 2005, are a stereo vision system and a radar system for obstacle detection. Offline simulation allows to analyze situations in the laboratory and to replay recordings from sensors. The Grand Challenge 2005 takes place on October 8, and the SciAutonics/Auburn team intends to compete with the improved RASCAL system.

Benchmarks of Low-Level Vision Algorithms for DSP, FPGA, and Mobile PC Processors

We present recent results of a performance benchmark of selected low-level vision algorithms implemented on different high-speed embedded platforms. The algorithms were implemented on a digital signal processor (DSP) (Texas Instruments TMS320C6414), a field-programmable gate array (FPGA) (Altera Stratix-I and II families) as well as on a mobile PC processor (Intel Mobile Core 2 Duo T7200). These implementations are evaluated, compared, and discussed in detail. The DSP and the mobile PC implementations, both making heavy use of processor-specific acceleration techniques (intrinsics and resource optimized slicing direct memory access on DSPs or Intel integrated performance primitives Library on mobile PC processors), outperform the FPGA implementations, but at the cost of spending all its resources to these tasks. FPGAs, however, are very well suited to algorithms that benefit from parallel execution.

Pfelib: a performance primitives library for embedded vision

This paper presents our work on PfeLib—a high performance software library for image processing and computer vision algorithms for an embedded system. The main target platform for PfeLib is the TMS320C6000 series of digital signal processors (DSPs) from Texas instruments. PfeLib contains several new approaches for problems that are typical when developing software for embedded systems. We propose a method for image data transfer from a development host (PC) to an embedded system for test and verification. This enables step-by-step performance optimizations directly on the target platform. An optimization procedure is described that illustrates our approach for obtaining the best possible DSP performance with a reasonable development effort. Speedup improvement factors of up to 16 were achieved. Also, the problem of the limited on-chip memory on DSPs is addressed by a novel double buffering method using direct memory access (DMA), called resource optimized slicing (ROS-DMA). ROS-DMA is intended to be used instead of L2 cache and it is a core component of PfeLib—it achieves up to six times faster image processing as compared to using L2 cache.

Extending the dynamic range of linear CMOS sensors

Extending the dynamic range of digital imagers is a rather recent applied science, where some research has already been done since Europe launched the joint research program EUREKA in 1986. However, the limitations of the image sensors and the processing power of the embedded processing systems used (e.g. in digital cameras), in addition to higher cost factors, prevented these systems being fast and therefore applicable. Since there is a growing demand for driving assistance systems and observation systems, special care and research is required to immunize vision systems against changing physical daylight conditions. This paper discusses a grayscale image sensor hardware framework and a recently developed implementation for the proposed solution to extend the dynamic range of a linear complementary metal-oxide-semiconductor (CMOS) sensor. A suggestion is made for a very large scale integration (VLSI) hardware system, which is firstly simulated in ModelSIM. In respect of generic attributes, the interfaces to image sensors and storage locations are highly adaptable in this approach.

Reliable architecture of an embedded stereo vision system for a low cost autonomous vehicle

In the 2005 DARPA Grand Challenge, five vehicles completed a preset course of 210 kilometers through desert dirt roads, completely driven by onboard automatic systems. This major achievement was accompanied by great progress of other vehicles which participated too but did not complete the course due to various reasons. The automatic vehicle RASCAL is one example of these vehicles: with its on-board autonomous capabilities, it reached a distance of 25.7 autonomously driven kilometers. One of the sensors implemented on RASCAL was a low-cost embedded stereo vision system. This dependable embedded system was based on a DSP platform and employed a novel software framework to guarantee reliable operations in the desert. This paper provides an overview on that particular subsystem