Hongtu Jiang

A Hardware Architecture for Real-Time Video Segmentation Utilizing Memory Reduction Techniques

This paper presents the implementation of a video segmentation unit used for embedded automated video surveillance systems. Various aspects of the underlying segmentation algorithm are explored and modifications are made with potential improvements of segmentation results and hardware efficiency. In addition, to achieve real-time performance with high resolution video streams, a dedicated hardware architecture with streamlined dataflow and memory access reduction schemes are developed. The whole system is implemented on a Xilinx field-programmable gate array platform, capable of real-time segmentation with VGA resolution at 25 frames per second. Substantial memory bandwidth reduction of more than 70% is achieved by utilizing pixel locality as well as wordlength reduction. The hardware platform is intended as a real-time testbench, especially for observations of long term effects with different parameter settings.

Hardware accelerator design for video segmentation with multi-modal background modelling

Among many of the algorithms for video segmentation, one based on a statistical background model (Stauffer, C. and Grimson, W., Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999) was developed with the unique feature of robustness in multi-modal background scenarios. However, with a large number of calculations due to the pixel-wise processing of each frame, such an algorithm could only achieve a low frame rate, far from real-time requirements, on computers. A hardware accelerator is proposed, with a dedicated architecture aimed at addressing both computation and memory bandwidth demands. The whole system is targeted to an FPGA platform, which serves as a real-time test bench where long term effects caused by fixed point quantization and various parameter settings can be studied. Meanwhile, memory bandwidth as well as memory size are investigated, and reduction by up to 60 percent, through similarity exploitation for neighboring Gaussian parameters, is envisioned. Furthermore, a controller synthesis tool is used to relieve the effort for the manual design of the complex control unit which schedules the operations of the whole system.

Real-Time Video Segmentation with VGA Resolution and Memory Bandwidth Reduction

This paper presents the implementation of a video segmentation unit used for embedded automated video surveillance systems. Various aspects of the underlying segmentation algorithm are explored and modifications are made with potential improvements of segmentation results and hardware efficiency. In addition, to achieve real-time performance with high resolution video streams, a dedicated hardware architecture with streamlined dataflow and memory access reduction schemes are developed. The whole system is implemented on a Xilinx FPGA platform, capable of real-time segmentation with VGA resolution at 25 frames per second. Substantial memory bandwidth reduction of more than 70% is achieved by utilizing pixel locality as well as wordlenghth reduction. The hardware platform is intended as a real-time testbench for observations of long term effects with different parameter settings, which is hard to achieve on a PC platform.

FPGA implementation of real-time image convolutions with three level of memory hierarchy

In this paper, a customized image convolution processor with three level memory hierarchy is implemented on Xilinx VirtexE FPGAs. Due to its fully pipelined datapath for calculations and streamlined data flow architecture, the processor has the performance close to that of TI highest performance C64x processor at less than 1/8 of the clock frequency with substantial I/O bandwidth reductions. Furthermore, potential power savings are envisioned in future ASIC implementations by meaningful memory hierarchy explorations. In addition, a dedicated controller composed of Finite State Machine with incremental branch optimization architecture is developed to control all the operations in calculations and data transfer.

FPGA implementation of controller-datapath pair in custom image processor design

In order to reduce the effort of the controller design in the customized image convolution processor, a controller synthesis tool is developed based on V. Owall (see ibid.,1994) to support the design flow from a system or algorithm specification to RTL level VHDL. Architecture extensions to basic FSMs structures are implemented with the purpose of optimizing controller design for area and power consumption. Together with controller implementation, a custom datapath architecture with three level memory hierarchies is developed aiming at a real-time power efficient image processing solution with low I/O bandwidth requirements. The complete design is prototyped in Xilinx Virtex 2 platform with comparable performance with that of TI C64x processor at only 2/15 of its clock frequency.