Seunghun Jin

FPGA Design and Implementation of a Real-Time Stereo Vision System

Stereo vision is a well-known ranging method because it resembles the basic mechanism of the human eye. However, the computational complexity and large amount of data access make real-time processing of stereo vision challenging because of the inherent instruction cycle delay within conventional computers. In order to solve this problem, the past 20 years of research have focused on the use of dedicated hardware architecture for stereo vision. This paper proposes a fully pipelined stereo vision system providing a dense disparity image with additional sub-pixel accuracy in real-time. The entire stereo vision process, such as rectification, stereo matching, and post-processing, is realized using a single field programmable gate array (FPGA) without the necessity of any external devices. The hardware implementation is more than 230 times faster when compared to a software program operating on a conventional computer, and shows stronger performance over previous hardware-related studies.

Design and Implementation of a Pipelined Datapath for High-Speed Face Detection Using FPGA

This paper presents design and implementation of a pipelined datapath for real-time face detection using cascades of boosted classifiers. We propose following methods: symmetric image downscaling, classifier sharing, and cascade merging, to achieve the desired processing speed and area efficiency. First, an image pyramid with 16 levels is generated from the input image to simultaneously detect faces with different scales. The downscaled images are then transferred to the first stage of the cascade that is shared between the corresponding image pairs based on the pixel validity of the symmetric image pyramid. The last method exploits the different hit ratios of the cascade stages. We use a tree-structured cascade of classifiers since most of the nonface elements are eliminated during the early stages of the classifier. The use of a synthesis tool confirms that the proposed design reduces resource utilization by one-eighth without accuracy loss, compared to the fully parallelized implementation of the same algorithm. We implemented the proposed hardware architecture on a Xilinx Virtex-5 LX330 FPGA. The indicative throughput is 307 frames/s irrespective of the number of faces in the scene for standard VGA (640 × 480) images with an operating frequency of 125.59 MHz. We may ensure that face detection results are generated at each clock cycle after the initial pipeline delay, using this fully pipelined datapath for tree-structured cascade classifiers.

An FPGA-based vehicle speed measurement system using an uncalibrated camera

Measuring a vehicles speed using a video image is an important technique in an intelligent transportation system. This paper presents a hardware implemented vehicle speed measurement (VSM) system. The operation of the system consists of three steps. First, we remove vehicles from a series of captured images to create a background. To do this, we introduce a simplified exclusion algorithm optimized to the hardware. Second, we subtract the background from the incoming image and concurrently apply affine rectification. Finally, based on the rectified image, we track the vehicles and measure the velocity. The system uses two SRAMs and one FIFO to store images. At each step, we present a memory management method of two SRAMs to store multiple images in one SRAM. This enables our system to obtain two images in a single full memory scan. Our system obtained ± 7.5% velocity error using an experiment comparing the vehicles speedometer and the vision-based measured velocity.

An FPGA-based Parallel Hardware Architecture for Real-Time Face Detection Using a Face Certainty Map

This paper presents an FPGA-based parallel hardware architecture for real-time face detection. An image pyramid with twenty depth levels is generated using the input image. For these scaled-down images, a local binary pattern transform and feature evaluation are performed in parallel by using the proposed block RAM-based window processing architecture. By sharing the feature look-up tables between two corresponding scaled-down images, we can reduce the use of routing resources by half. For prototyping and evaluation purposes, the hardware architecture was integrated into a Virtex-5 FPGA. The experimental result shows around 300 frames per second speed performance for processing standard VGA (640×480×8) images. In addition, the throughput of the implementation can be adjusted in proportion to the frame rate of the camera, by synchronizing each individual module with the pixel sampling clock.

Real-time sound source localization system based on FPGA

This paper presents a real-time sound source localization system based on FPGA (field programmable gate array). The system employs three microphones with different locations and calculates the direction of the sound source using TDOA (time-delay of arrivals). All the functions for the sound source localization are implemented with the use of a dedicated parallel architecture, including acoustic signal capturing, cross correlation, short-term energy and azimuth computation. The proposed system uses two different clocks for achieving real-time performance - low frequency signal capturing clock and high frequency processing clock. The proposed system can be used to intelligent service robots, security and surveillance applications as a result of a FPGA implementation.

Automatically available photographer robot for controlling composition and taking pictures

Recent advances made in IT technology has given much impetus to the development of multimedia devices. The digital camera is such a multimedia device. It has made much progress and become very popular, as most people now own a digital camera or cell phone with camera features. People often take photographs in everyday life. Professional photographers often take photographs of travel destinations, banquet halls or parties. In this paper, we propose an autonomous robot photographer capable of taking pictures and thus replacing photographers. This photographer robot can detect direction based on the human voice. It can control composition based on skin color detection to snap the picture.

A dedicated hardware architecture for real-time auto-focusing using an FPGA

The auto-focus is a fundamental function of a camera system which is required to photograph a clear image of an object. To obtain the optimal focus of a specific region within an image, the sharpness of the region must be measured. Since the sharpness represents the difference between a pixel and its neighbors, multiple pixel references occur while evaluating each pixel. To compensate for the processing bottleneck caused by this repetitive memory reference, this paper presents a dedicated hardware architecture for real-time auto-focusing. The proposed system processes the incoming pixel simultaneously with its neighboring pixels based on its parallelized window processing architecture. In addition, the proposed system performs an adaptive thresholding-based sharpness function with multiple windows to achieve accuracy and robustness. The proposed system is compared to several conventional pixel-based auto-focusing systems under various environmental conditions.

Pipelined Hardware Architecture for High-Speed Optical Flow Estimation Using FPGA

Optical flow is a motion field estimation method that has a wide range of applications. In this paper, we present a fully pipelined hardware architecture for high-speed optical flow estimation based on a full-search block matching algorithm. A census transform is applied to the corresponding pixels in the current and previous frame. The similarity between two census vectors within the search area is then computed by measuring the hamming distance. Macro blocks are generated based on the measured hamming distance values and the best match is determined by locating the block that has the smallest sum. The synthesis tool reported that the proposed system is capable of processing 400 standard VGA frames per second.

A Real-Time Histogram Equalization System with Automatic Gain Control Using FPGA

High quality camera images, with good contrast and intensity, are needed to obtain the desired information. Images need to be enhanced when they are dark or bright. The histogram equalization technique, which flattens the density distribution of an image, has been widely used to enhance image contrast due to its effectiveness and simplicity. This technique, however, cannot be used to enhance images that are either too dark or too bright. In addition, it is difficult to perform histogram equalization in real-time using a general-purpose computer. This paper proposes a histogram equalization technique with AGC (Automatic Gain Control) to extend the image enhancement range. It is designed using VHDL (VHSIC Hardware Description Language) to enhance images in real-time. The system is implemented with an FPGA (Field Programmable Gate Array). An image processing system with this FPGA is implemented. The performance of this image processing system is measured.

FPGA-based image processing system for remote robot control

Image-based control, using a mobile terminal, is a widely used control method for many remote robot applications since it provides valuable visual information from around the remote robot to the operator. However, the computational power of the mobile terminal is normally limited, compared with that of the stationary computing machine, because it has a number of constraints such as size, power, and cost. Since the mobile robot also has its inherent tasks, processing a large amount of data may cause other performance problems as the intelligence level grows. For this reason, it is not good practice to perform image processing via software programs, either in the mobile terminal or the in remote robot. This paper proposes a dedicated hardware architecture which can assist both the mobile terminal and the remote robot by taking complete charge of the vision-related tasks and thus decreasing the computational burden still to be performed. As a result, the remote robot can fully use its computation power for its main tasks so that the overall performance and efficiency can be increased.