Christopher T. Johnston

FPGA implementation of a Single Pass Connected Components Algorithm

The classic connected components labelling algorithm requires two passes through an image. This paper presents an algorithm that allows the connected components to be analysed in a single pass by gathering data on the regions as they are built. This avoids the need for buffering the image, making it ideally suited for processing streamed images on an FPGA or other embedded system with limited memory. An FPGA-based implementation is described, emphasising the modifications made to the algorithm to enable it to satisfy timing constraints.

Optimised single pass connected components analysis

Classical connected components labelling algorithms are unsuitable for real-time processing of streamed images on an FPGA because they require two passes through the image. Recently, a single-pass algorithm was proposed that avoided the need to buffer an intermediate image. In this paper, a new single pass algorithm is described that is a considerable improvement over the existing algorithms. The new algorithm reassigns and reuses labels each row to minimise the size of both the equivalence and region data tables. The optimised single-pass algorithm reduces the worst case memory requirement by over 100 times that of the original algorithm (for measuring region area), and reduces the latency to only 1 row.

Connected components analysis of streamed images

Classical connected components labelling algorithms are unsuitable for real-time processing of streamed images on an FPGA because they require two passes through the image. The basic requirements of connected components analysis are investigated, and this led to a novel single-pass approach that avoids the need to buffer an intermediate image. This is further analysed to give an algorithm that is both memory efficient and has the minimum latency. The final algorithm reduces the memory by over 300 times, and reduces the latency by more than a factor of 2.

A visual environment for real-time image processing in hardware (VERTIPH)

Real-time video processing is an image-processing application that is ideally suited to implementation on FPGAs. We discuss the strengths and weaknesses of a number of existing languages and hardware compilers that have been developed for specifying image processing algorithms on FPGAs. We propose VERTIPH, a new multiple-view visual language that avoids the weaknesses we identify. A VERTIPH design incorporates three different views, each tailored to a different aspect of the image processing system under development; an overall architectural view, a computational view, and a resource and scheduling view.

Design Patterns for Image Processing Algorithm Development on FPGAs

FPGAs are often used as implementation platforms for real-time image processing applications because their structure allows them to exploit spatial and temporal parallelism. Such parallelization is subject to the processing mode and hardware constraints including limited processing time, limited access to data and limited resources of the system. These constraints often force the designer to reformulate the software algorithm in the process of mapping it to hardware. To aid in the process this paper proposes the application of design patterns which embody experience and through reuse provide tools for solving particular mapping problems. Issues involved in applying design patterns in this manner are outlined and discussed.

Using design patterns to overcome image processing constraints on FPGAs

The mapping of image processing algorithms to hardware is complicated by several hardware constraints including limited processing time, limited access to data and limited resources of the system. These constraints often force the designer to reformulate the algorithm. To aid in the process this paper details the application of design patterns to image processing algorithm development. Design patterns embody experience and through reuse provide tools for solving particular mapping problems. The effectiveness of design patterns for overcoming constraints in the mapping process is illustrated in the context of a real world example that focuses on the development of a real-time object tracking algorithm implemented on an FPGA.

Algorithm Transformation for FPGA Implementation

High level hardware description languages aim to make hardware design more like programming software. These languages are often used to accelerate legacy software algorithms by porting them to an FPGA based hardware implementation. Porting does not always result in efficient architectures as the original algorithms are usually developed and optimized to run on a serial processor. To obtain an efficient hardware architecture, one that makes use of the available parallelism, the algorithms need to be transformed. Eleven such transformations are identified and explained. While some of these are straightforward, and have been implemented by some compilers, many cannot be automated because they require detailed knowledge of the algorithm.

User evaluation and overview of a visual language for real time image processing on FPGAs

FPGAs are often used for image processing, but existing FPGA design tools lack syntactic constructs for some specialized activities that are important in this field, such as timing, resource handling and scheduling. This forces the developer to work at too low a level and makes it difficult to produce a genuinely hierarchically decomposed design. This paper outlines these deficiencies, as the background for an overview of and justification for each of three views in VERTIPH, a visual programming language for defining image processing algorithms on FPGAs. This updates the overview presented in [1]. The paper then presents the results of two user evaluations of VERTIPH, a pre-implementation paper-based user evaluation which found no major changes were required and a post-(partial)-implementation user evaluation. The latter evaluated the novel parts of the language using participants experienced in the field. The key parts of VERTIPH were found to be useful visualisations for the developers, and the only major problem was the interaction required for defining type-connections between views.

GATOS: a windowing operating system for FPGAs

FPGAs are increasingly being used to implement low-level vision operations in stand-alone configurations. For real-time image processing applications there is a need for several interactive operating system functions such as tuning, calibration, user interaction, and debugging which are difficult to perform when not under the control of a host operating system. This paper proposes the gate array terminal operating system (GATOS) which provides a set of high-level IP blocks that implement the graphical user interface of an interactive windowing operating system. The requirements and desired levels of functionality for GATOS are discussed and the preliminary design is presented.

Towards a visual notation for pipelining in a visual programming language for programming FPGAs

VERTIPH is a visual language designed to aid in the development of image processing algorithms on FPGAs (Field Programmable Gate Arrays). We justify the use of a visual language for this purpose, and describe the key parts of VERTIPH. One aspect of importance is how to clearly and efficiently represent a pipeline of processors, and in particular distinguish a pipeline from the simpler serial or parallel structures. This paper develops a number of pipeline representations, discussing the rationale behind and limitations associated with each representation. The culmination of this development is the Sequential Pipeline with Detailed Bars, visually an efficient and unambiguous representation.