Andy Motten

Binary confidence evaluation for a stereo vision based depth field processor SoC

This paper presents a methodology to construct a binary confidence value for every pixel of a depth map. We start by constructing 72 different confidence metrics, including the traditional ones and new metrics based on neighborhood information. Construction of the binary confidence value from these metrics is hence viewed as a two-class classification problem where we evaluated three different classifiers, with increasing complexity. Only metrics and classifiers that are suitable for VLSI hardware implementation will be evaluated. Evaluation of the constructed classifiers is performed on an indoor dataset of Stereo Images.

Low-cost real-time stereo vision hardware with binary confidence metric and disparity refinement

This paper presents a real-time stereo vision System-on-Chip (SoC) architecture for a depth-field generation processor as required in 3D TV applications. This architecture includes post-processing steps like a decision tree based confidence metric and a disparity refinement module while still fitting in a low cost FPGA. A real-time stereo matching calculation at a frame rate of 56 Hz with a resolution of 800x600 and a disparity of 80 has been realized using this architecture without the need for external memories.

A binary adaptable window SoC architecture for a stereo vision based depth field processor

This paper presents a novel binary fully adaptable window for incorporating in a stereo matching System-on-Chip (SoC) architecture. This architecture is fully scalable and parameterizable to allow for custom SoC implementations, as well as rapid prototyping on FPGAs. For each window a binary mask window is generated which selects the supporting pixels in the cost aggregation phase of the SAD algorithm. This selection is performed using color similarity and spatial distance metrics. Hardware resource utilization for a fixed window and an adaptable window cost aggregation is compared based on FPGA logic element use.

Adaptive memory architecture for real-time image warping

This paper presents a real time image warping module implemented in hardware. A look-up table (LUT) based reverse mapping is used to relate the source image to the warped image. Frame buffers or line buffers are often used to temporally store the source image. However these methods do not take the underlying pattern of the reverse mapping coordinates into account. The presented architecture uses an adaptable memory allocation which can change the depth and the position of the line buffer between lines. A real-time stereo rectification use case has been implemented to validate the operation of this module. Depending on the scenario, the memory consumption can be reduced by a factor of two and more. A real-time image warping module for video cameras has been implemented in a single FPGA, without the use of off-chip memories.

Trinocular Stereo Vision Using a Multi Level Hierarchical Classification Structure

A real-time trinocular stereo vision processor is proposed which combines a window matching architecture with a classification architecture. A pair wise segmented window matching for both the center-right and center-left image pairs as their scaled down image pairs is performed. The resulting cost functions are combined which results into nine different cost curves. A multi level hierarchical classifier is used to select the most promising disparity value. The classifier makes use of features provided by the calculated cost curves and the pixels’ spatial neighborhood information. Evaluation and classifier training has been performed using an indoor dataset. The system is prototyped on an FPGA board equipped with three CMOS cameras. Special care has been taken to reduce the latency and the memory footprint.

An on-chip parallel memory architecture for a stereo vision system

This paper presents a novel parallel System-on-Chip (SoC) memory architecture for a stereo vision system as required in 3D TV applications. It allows for a parallel access to all pixels located in a chosen window of the image. Using this architecture a complete window refresh on each clock cycle is possible, which can be used to increase the depth range of a stereo vision algorithm. This architecture is fully scalable and parameterizable to allow for custom SoC implementations, as well as rapid prototyping on FPGAs. Hardware resource utilization for different processor window size configurations is compared based on FPGA logic element use.

Smart camera SoC system for interactive real-time real-brush based digital painting systems

Advanced digital paint systems, based on accurate simulation models for the paint, brush and canvas interaction enable a virtual painting environment familiar to artists, but also allow for new ways of expressivity. The new FluidPaint co-located real-brush-input & canvas-display systems put extremely high demands on the real-time image processing and the reaction time between brush strokes and the rendering of the painting result on the screen. A new Smart Camera System-on-Chip (SoC) architecture is presented enabling real-time input of brush-strokes with real brushes in the FluidPaint digital painting system.

Smart Camera System-on-Chip Architecture for Real-Time Brush Based Interactive Painting Systems

Interactive virtual paint systems are very useful in editing all kinds of graphics artwork. Because of the digital tracking of strokes, interactive editing operations such as save, redo, resize etc. are possible. The structure of artwork generated can be used for animation in artwork cartoons. A novel System-on-Chip Smart Camera architecture is presented that can be used for tracking infrared fiber based brushes as well as real brushes in real-time. A dedicated SoC hardware implementation avoids unnecessary latency delays caused by PC based architectures, that require communication-, PC and GPU frame-buffer delays, thereby considerably enhancing the interactivity experience. The system is prototyped on an FPGA.