Wenhao He

An improved Canny edge detector and its realization on FPGA

Canny edge detector treats edge detection as a signal processing problem to design an optimal edge detector and has been widely used for edge detection. However, the traditional Canny edge detector has two shortcomings. First, the threshold of the algorithm needs to be set by manual. Secondly, the algorithm is very time consuming and can not be implemented in real time. A new self-adapt threshold Canny algorithm is proposed in this paper to solve the first problem. A pipelined implementation on FPGA for this new algorithm is also designed to solve the second problem. Experiment results are also given to show the efficiency of the proposed method.

Dynamic and tip-over stability analysis of a planetary wheeled stair-climbing wheelchair

In this paper, the design of the mechanism and the control system of a planetary wheeled stair-climbing wheelchair is introduced. The dynamic model of the planetary wheel clusters is established based on Lagrange equation, and the angle acceleration curves are studied with various given equivalent torques. Stability margin of the wheelchair is analyzed in detail during a single step climbing procedure. According to the simulation results, the control law of wheelchairs angle is derived from the projection of the wheelchairs CG on the condition that the wheelchairs stability is always maintained. The wheelchair can be easily operated by an assistant accordance with the control law.

Real-time scene recognition on embedded system with SIFT keypoints and a new descriptor

The vision system of a mobile robot has to interpret the environment in real time at low power. As a good algorithm for extracting information from images, SIFT (Scale Invariant Feature Transform) is widely used in computer vision. However, the high computational complexity makes it hard to achieve real-time performance of SIFT with pure software. This paper presents a machine vision system implementing the SIFT algorithm on an embedded image processing card, where real-time scene recognition is accomplished with low power consumption through the cooperation between an FPGA (Field Programmable Gate Array) and a DSP (Digital Signal Processor) chip. The original SIFT keypoint detection algorithm is adapted for parallel computation and implemented with a hardware pipeline in the FPGA. Although our current system is designed for 360×288 video frames, this pipelined architecture can be applied to images with arbitrary resolution. Meanwhile, the original 128-dimensional SIFT descriptor is replaced by an 18-dimensional new descriptor which can be generated more efficiently and can be matched according to an absolute distance threshold with the distance defined by infinity-norm. On this basis, a five-branch-tree data structure is designed for fast searching and matching of descriptors, and robust scene recognition is realized through the combination of keypoints. Since our new descriptor allows one keypoint to be matched to several keypoints, which is a distinct property from the original SIFT algorithm, our system can recognize multiple images with overlapping contents simultaneously. In addition, compared with traditional work that needs off-line training, our system can perform fast on-line learning, which is a desirable property for mobile robots.

A high speed robot vision system with GigE vision extension

High speed image and video processing is becoming increasingly important in many applications, especially in robotics. To boost the computing speed of traditional robot vision system, a FPGA and DSP based robot vision system is developed. Considering about the high throughput image acquisition is the premise of high speed processing, a GigE vision interface is also extended. The configuration and some important characteristics of this robot vision system, which can not only capture images rapidly but can also process images using different algorithms in real-time, are described in this paper. Experiment results are also given to show that the newly developed vision system is much faster and more suitable for robot vision applications.

A fast stereovision measurement algorithm based on SIFT keypoints for mobile robot

When a robot moves independently, it needs to localize itself and avoid obstacles. In order to enable the robot to obtain the depths of surrounding objects, a fast stereovision measurement algorithm based on SIFT keypoints is proposed and implemented on an embedded image processing board based on DSP and FPGA. The calculation of the keypoint descriptor is simplified by directly using the pixel values in the block neighborhood of a keypoint, resulting in an improvement in real-time performance. In order to guarantee the accuracy of searching for the matching points, a two-pass searching strategy is employed, that is, to choose a point in the left image and search the candidate point in the right image, then determine the correspondence by searching the matching point in the left image again. Moreover, for improving the precision of the measurement results, a quadratic polynomial is introduced to compensate the measurement results. Measurement results and comparison of different methods demonstrate the effectiveness of the proposed algorithm. In addition, the measurement system is applied to a mobile robot platform, and the experiment results validate that the system can satisfy the demand of robot applications.

Real-time SIFT-based object recognition system

In this paper a real-time object recognition system is realized, based on the Scale Invariant Feature Transform (SIFT) algorithm. The system mainly contains a display, a camera and an image acquisition and processing board developed by our research team. An FPGA chip and a DSP chip are embedded in the card as the major calculation units, which make real-time computation possible. The whole recognition algorithm is divided into three parts: the detection of SIFT keypoints, the extraction of SIFT descriptors and the final object recognition. In order to achieve real-time detection of SIFT keypoints through hardware computation on FPGA, the original SIFT algorithm is adapted to accommodate the parallel computation and pipelined structure of hardware. Using a mode of DSP invoking a customized FPGA module, a 72-dimensional keypoint descriptor is proposed to save memory space and to cut down the computing cost in keypoints matching. The recognition proceeds by matching individual features to a database of features from known objects using a fast approximate nearest-neighbor search algorithm changed based on the k-d tree and the BBF algorithm. In addition, three matching strategies are adopted to discard the false matches so as to improve the accuracy of recognition. The object recognition functionality is mainly achieved in the DSP. A model database is built and used to test the accuracy and effectiveness of the system.

An improvement on the keypoint detection method of SIFT for hardware computation

In order to achieve real-time detection of SIFT keypoints through hardware computation on FPGA, the original algorithm was redesigned to accommodate the parallel computation and pipelined structure of hardware. The computation accuracy of fixed-point number is improved by the new scheme, while the computation amount of the whole algorithm is greatly reduced and hardware cost is saved. In the aspect of performance, the new scheme is as robust to image noise as the original algorithm, while the scale invariance of keypoints has been improved dramatically.

A real-time small immobile object recognition system using wavelet moment invariants

In this paper, a real-time small immobile object recognition system is implemented using wavelet moment-based back-propagation(BP) neural network classifier. The system is composed of a camera and an image acquiring and processing board developed by our research team. An FPGA chip and a DSP chip are embedded in the image board as the major calculation units, which make real-time computation possible. First, wavelet moment invariants of training samples are integrated with BP neural network to construct the classifier on the host computer. Then, real-time object detection and classification experiments are conducted according to the classifier on the image acquiring and processing board. Experiment results show that the algorithm can detect and classify different small immobile object types efficiently.

A Real-time Small Moving Object Detection System Based on Infrared Images

In this paper a real-time small moving object detection system is realized based on infrared image. The system is composed of a display, a camera and an image acquisition and processing card developed by our research team. An FPGA chip and a DSP chip are embedded in the image card as the major calculation units, which make real-time computation possible. An efficient object detection algorithm is customized for this system, consisting of two stages: the extraction of suspicious objects based on the single frame and the detection of real objects based on the image sequences. The first stage comprises image smoothing and morphological operations which are carried out in the FPGA, while the second stage contains connected component analysis and movement analysis which are implemented in the DSP. In the latter stage, a ring pointer cache structure is designed in order to save memory and speed up the processing, and three integer parameters are used so as to index all the images and objects quickly. In addition, a fast matching algorithm is presented to string the candidate objects in adjacent frames for movement analysis. Finally, two experiments are conducted. Firstly, limited by experimental conditions, a low-quality video taken by a low cost infrared camera is used to test the effectiveness of the algorithm. Secondly, an artificial simulation scenario is built to test the accuracy and real-time performance of the embedded system.

Line object extraction and image matching based on an intelligent image card

A robot vision system is developed based on an intelligent image gathering card that contains an FPGA (Field Programmable Gate Array) and a DSP (Digital Signal Processor) as main calculators. Two real-time visual modules are developed through the cooperation of hardware logics on FPGA and software on DSP. First, with edge images extracted by the FPGA, a highly efficient algorithm is designed to extract a line object on the ground with DSP on real-time. Second, an image matching algorithm based on gray variance forms is designed, which is robust to illumination changing. Real-time computation is achieved by implementing this algorithm on FPGA, and object recognition is realized further.