Hangen He

Visual Saliency Based on Scale-Space Analysis in the Frequency Domain

We address the issue of visual saliency from three perspectives. First, we consider saliency detection as a frequency domain analysis problem. Second, we achieve this by employing the concept of nonsaliency. Third, we simultaneously consider the detection of salient regions of different size. The paper proposes a new bottom-up paradigm for detecting visual saliency, characterized by a scale-space analysis of the amplitude spectrum of natural images. We show that the convolution of the image amplitude spectrum with a low-pass Gaussian kernel of an appropriate scale is equivalent to an image saliency detector. The saliency map is obtained by reconstructing the 2D signal using the original phase and the amplitude spectrum, filtered at a scale selected by minimizing saliency map entropy. A Hypercomplex Fourier Transform performs the analysis in the frequency domain. Using available databases, we demonstrate experimentally that the proposed model can predict human fixation data. We also introduce a new image database and use it to show that the saliency detector can highlight both small and large salient regions, as well as inhibit repeated distractors in cluttered images. In addition, we show that it is able to predict salient regions on which people focus their attention.

Saliency Detection Based on Frequency and Spatial Domain Analyses.

We propose a new saliency detection model by combining global information from frequency domain analysis and local information from spatial domain analysis. In the frequency domain analysis, instead of modeling salient regions, we model the nonsalient regions using global information; these so-called repeating patterns that are not distinctive in the scene are suppressed by using spectrum smoothing. In spatial domain analysis, we enhance those regions that are more informative by using a center-surround mechanism similar to that found in the visual cortex. Finally, the outputs from these two channels are combined to produce the saliency map. We demonstrate that the proposed model has the ability to highlight both small and large salient regions in cluttered scenes and to inhibit repeating objects. Experimental results also show that the proposed model outperforms existing algorithms in predicting objects regions where human pay more attention.

A hierarchical approach for road detection

Road detection is a crucial problem for autonomous navigation system (ANS) and advance driver-assistance system (ADAS). In this paper, we propose a hierarchical road detection method for robust road detection in challenging scenarios. Given an on-board road image, we first train a Gaussian mixture model (GMM) to obtain road probability density map (RPDM), and next oversegment the image into superpixels. Based on RPDM and superpixels, initial seeds are selected in an unsupervised way, and the seed superpixels iteratively try to occupy their neighbors according to GrowCut framework, the road segment is obtained after convergency. Finally, we refine the road segment with a conditional random field (CRF), which enforces the shape prior on the road segmentation task. Experiments on two challenging databases demonstrate that the proposed method exhibits high robustness compared with the state-of-the-art.

Real-time lane departure warning system based on a single FPGA

AbstractThis paper presents a camera-based lane departure warning system implemented on a field programmable gate array (FPGA) device. The system is used as a driver assistance system, which effectively prevents accidents given that it is endowed with the advantages of FPGA technology, including high performance for digital image processing applications, compactness, and low cost. The main contributions of this work are threefold. (1) An improved vanishing point-based steerable filter is introduced and implemented on an FPGA device. Using the vanishing point to guide the orientation at each pixel, this algorithm works well in complex environments. (2) An improved vanishing point-based parallel Hough transform is proposed. Unlike the traditional Hough transform, our improved version moves the coordinate origin to the estimated vanishing point to reduce storage requirements and enhance detection capability. (3) A prototype based on the FPGA is developed. With improvements in the vanishing point-based steerable filter and vanishing point-based parallel Hough transform, the prototype can be used in complex weather and lighting conditions. Experiments conducted on an evaluation platform and on actual roads illustrate the effective performance of the proposed system.

Robust Unstructured Road Detection: The Importance of Contextual Information

Unstructured road detection is a key step in an unmanned guided vehicle (UGV) system for road following. However, current vision-based unstructured road detection algorithms are usually affected by continuously changing backgrounds, different road types (shape, colour), variable lighting conditions and weather conditions. Therefore, a confidence map of road distribution, one of contextual information cues, is theoretically analysed and experimentally generated to help detect unstructured roads. Two traditional algorithms, support vector machine (SVM) and k-nearest neighbour (KNN), are carried out to verify the helpfulness of the proposed confidence map. Following this, a novel algorithm, which combines SVM, KNN and the confidence map under a Bayesian framework, is proposed to improve the overall performance of the unstructured road detections. The proposed algorithm has been evaluated using different types of unstructured roads and the experimental results show its effectiveness.

Robust curb detection with fusion of 3D-Lidar and camera data.

Curb detection is an essential component of Autonomous Land Vehicles (ALV), especially important for safe driving in urban environments. In this paper, we propose a fusion-based curb detection method through exploiting 3D-Lidar and camera data. More specifically, we first fuse the sparse 3D-Lidar points and high-resolution camera images together to recover a dense depth image of the captured scene. Based on the recovered dense depth image, we propose a filter-based method to estimate the normal direction within the image. Then, by using the multi-scale normal patterns based on the curbs geometric property, curb point features fitting the patterns are detected in the normal image row by row. After that, we construct a Markov Chain to model the consistency of curb points which utilizes the continuous property of the curb, and thus the optimal curb path which links the curb points together can be efficiently estimated by dynamic programming. Finally, we perform post-processing operations to filter the outliers, parameterize the curbs and give the confidence scores on the detected curbs. Extensive evaluations clearly show that our proposed method can detect curbs with strong robustness at real-time speed for both static and dynamic scenes.

Ribbon Model based path tracking method for autonomous land vehicle

To address the path tracking problem of autonomous land vehicle, a new vehicle-road model named “Ribbon Model” is constructed under the constraints of road width and vehicle geometry structure. A new vehicle-road evaluation algorithm is developed based on this model, and new path tracking controller is designed. The difficulties of preview distance selection and parameters tuning with speed of pure following controller are avoided in this controller. Performance of the novel method is verified by simulation and vehicle experiments.

Lane Detection Using Steerable Filters and FPGA-based Implementation

Vision-based lane detection is a key component for Driver-Assistance (DA) systems. It is still a challenging task in road scenes with complex shadows. This paper presents a novel local edge detector, using vanishing point position as a high level information to guide the use of steerable flters in lane detection, and its implementation on a Field Programmable Gate Array (FPGA) device. The FPGA technology has the advantages of high-performances for digital image processing and low cost, both of which are the requirements of DA systems. The main contributions of this work are twofold: 1) an edge extraction algorithm for lane detection is proposed, using the estimated vanishing point as high-level information to detect lanes. Firstly, a rough estimation of the vanishing point is used for calculating the expected local edge orientations. Secondly, a steerable flter is tuned to the expected direction for edge response. 2) a framework on FPGA is designed to implement the proposed algorithm. The framework is designed by using multi-engine technology, so it works in parallel for any order of steerable flters. Experiments and comparisons show that the proposed algorithm is very effcient in dealing with the complex shadow conditions, and works in real-time on FPGA device.

A real-time lane departure warning system based on FPGA

This paper presents a vision based Lane Departure Warning System (LDWS) and its implementation on a Field Programmable Gate Array (FPGA) device. It is used as a Driver Assistance (DA) system that supports drivers and helps avoiding accidents. The FPGA technology has the advantages of high-performances for digital image processing and low cost, both of which are the requirements of the DA systems. The main contributions of this work are threefold: 1) a hardware architecture, which combines Single Instruction Multiple Data (SIMD) structure and Single Instruction Single Data (SISD) structure based on FPGA, is implemented. This architecture is in possession of both efficiency and flexibility. Therefore, it can be employed to handle many vision processing tasks in real time; 2) an improved parallel Hough Transform (HT) is introduced. Compared with traditional HT, we move the origin to the estimated vanishing point, so as to reduce the storage requirements and improve the detection robustness; and 3) a simple and efficient warning strategy is presented which can be implemented on FPGA easily. Experiments illustrate the high performance of the introduced system in various common roadway scenes.

Vision Sensor-Based Road Detection for Field Robot Navigation

Road detection is an essential component of field robot navigation systems. Vision sensors play an important role in road detection for their great potential in environmental perception. In this paper, we propose a hierarchical vision sensor-based method for robust road detection in challenging road scenes. More specifically, for a given road image captured by an on-board vision sensor, we introduce a multiple population genetic algorithm (MPGA)-based approach for efficient road vanishing point detection. Superpixel-level seeds are then selected in an unsupervised way using a clustering strategy. Then, according to the GrowCut framework, the seeds proliferate and iteratively try to occupy their neighbors. After convergence, the initial road segment is obtained. Finally, in order to achieve a globally-consistent road segment, the initial road segment is refined using the conditional random field (CRF) framework, which integrates high-level information into road detection. We perform several experiments to evaluate the common performance, scale sensitivity and noise sensitivity of the proposed method. The experimental results demonstrate that the proposed method exhibits high robustness compared to the state of the art.