Chunzhao Guo

Robust Road Detection and Tracking in Challenging Scenarios Based on Markov Random Fields With Unsupervised Learning

This paper presents a robust stereo-vision-based drivable road detection and tracking system that was designed to navigate an intelligent vehicle through challenging traffic scenarios and increment road safety in such scenarios with advanced driver-assistance systems (ADAS). This system is based on a formulation of stereo with homography as a maximum a posteriori (MAP) problem in a Markov random held (MRF). Under this formulation, we develop an alternating optimization algorithm that alternates between computing the binary labeling for road/nonroad classification and learning the optimal parameters from the current input stereo pair itself. Furthermore, online extrinsic camera parameter reestimation and automatic MRF parameter tuning are performed to enhance the robustness and accuracy of the proposed system. In the experiments, the system was tested on our experimental intelligent vehicles under various real challenging scenarios. The results have substantiated the effectiveness and the robustness of the proposed system with respect to various challenging road scenarios such as heterogeneous road materials/textures, heavy shadows, changing illumination and weather conditions, and dynamic vehicle movements.

Stereovision-based road boundary detection for intelligent vehicles in challenging scenarios

Road detection is a crucial problem for intelligent vehicles and mobile robots. Most of the methods proposed nowadays only achieve reliable results in relatively well-arranged environments. In this paper, we proposed a stereovision-based road boundary detection method by combining homography estimation and MRF-based belief propagation to cope with challenging scenarios such as unstructured roads with unhomogeneous surfaces. In the method, each pixel in the reference image is firstly labeled as “road” or “non-road” by minimizing a well defined energy function that accounts for the planar road region. Subsequently, both of the road boundaries are generated using Catmull-Rom splines based on RANdom SAmple Consensus (RANSAC) algorithm with varying road structure models to help the intelligent vehicle understand the structure as well as safe range of current road. In the suggested framework, both intensity and geometry information of road scenarios are used to contain all the regions belonging to the planar road plane, and the left and right road boundaries are generated separately using a robust fitting algorithm to handle different road structures. Therefore, more accurate as well as robust detection of the road can be expected. Experimental results on a wide variety of typical but challenging scenarios have demonstrated the effectiveness of the proposed method.

Lane detection and tracking in challenging environments based on a weighted graph and integrated cues

Real-time lane detection and localization is one of the key issues for many intelligent transportation systems. In this paper, we present a lane detection and tracking approach designed to work in challenging environments where lane boundaries may be low-contrast and changeful with noise due to a number of factors such as wear, type, lighting and weather conditions, etc. In the method, a sophisticated cascade lane feature detector is applied to cope with challenging environments at the very beginning of the detection and a weighted graph is subsequently constructed by integrating intensity as well as geometry cues, reflecting the confidence of each pixel as a lane feature. In order to deal with complex road geometry, we employ Catmull-Rom splines to represent lane boundaries and the left and right lane boundaries are estimated separately in a tracking process using particle filter based on the weighted graph. In the proposed framework, unlike most of previous methods we lay a strong emphasis on accurate and effective lane feature detection since the challenges happen in the very first step of lane detection, and accurately detected lane features can be expected to reduce the complexity and difficulty, as well as improve the accuracy of lane detection in the following steps.

Automatic lane-level map generation for advanced driver assistance systems using low-cost sensors

Lane-level digital maps can simplify driving tasks for robotic cars as well as enhance performance and reliability for advanced driver assistance systems (ADAS) by providing strong priors about the driving environment. In this paper, we present a system for automatic generation of precise lane-level maps by using conventional low-cost sensors installed in most of current commercial cars. It mainly consists of two modules, i.e. road orthographic image generation and lane graph construction. First, we divide the global map into fixed local segments based on the road network topology. According to the local map segments, we accumulate the birds eye view images of the road surface by fusing GPS, INS and visual odometry, and subsequently integrate them into synthetic orthographic images with the reference of the local map segments. Furthermore, the information of the driving lanes is extracted from the orthographic images and a large amount of vehicle trajectories, which is used to construct the lane graph of the map based on the lane models we proposed. Such a system can offer increased value as well as promote the automation level for todays commercial cars without being supplemented additional sensors. Experiments show promising results of the automatic map generation of the real-world roads, which substantiated the effectiveness of the proposed approach.

A Multimodal ADAS System for Unmarked Urban Scenarios Based on Road Context Understanding

Comprehensive situational awareness is paramount to the effectiveness of advanced driver assistance systems (ADASs) used in daily urban traffic, particularly for the unmarked roads, which cannot fulfill the requirements of conventional ADAS systems. This paper proposed a stereovision-based multimodal ADAS system designed for expanding the usability of ADAS functions, including lane-keeping assist, adaptive cruise control, and precrash system, to normal urban scenarios with unmarked roads. At first, the physical road boundary and vehicle candidates are detected. Subsequently, the contextual information between the host vehicle, the road, and the other vehicles are correlated for both low-level object detection improvement and high-level road structure estimation. Finally, the required ADAS elements are generated based on the correlation results with respect to the system functionalities. Experimental results in various typical but challenging scenarios have substantiated the effectiveness of the proposed system, which could help increase the value of the existing ADAS system without major modifications or expense.

Drivable road region detection using homography estimation and efficient belief propagation with coordinate descent optimization

Road detection is one of the key issues for the implementation of intelligent vehicles. In this paper, we present a drivable road region detection method using homography estimation and efficient belief propagation. In the method, each pixel in stereo images is assigned a label by minimizing an energy function that accounts for the planar road region, which is defined by utilizing the 2D projective transformations of stereo information and the inference algorithm in binary piecewise Markov Random Field (MRF). The energy is minimized in coordinate descent iterations that alternate between optimizing the homography induced by the planar road plane and implementing efficient belief propagation to find the optimal binary labeling that segments the image into two non-overlapping road and non-road regions. In the optimization process, both image evidence and temporal information are used; meanwhile, an error correction mechanism is applied. Therefore, more accurate as well as robust detection of the road region can be expected. Experimental results on a wide variety of typical but challenging real road scenes have substantiated the effectiveness as well as robustness of the proposed method.

Probabilistic Inference for Occluded and Multiview On-road Vehicle Detection

Visual-based approaches have been extensively studied for on-road vehicle detection; however, it faces great challenges as the visual appearance of a vehicle may greatly change across different viewpoints and as a partial observation sometimes happens due to occlusions from infrastructure or scene dynamics and/or a limited camera vision field. This paper presents a visual-based on-road vehicle detection algorithm for a multilane traffic scene. A probabilistic inference framework based on part models is proposed to overcome the challenges from a multiview and partial observation. Geometric models are learned for each dominant viewpoint to describe the configuration of vehicle parts and their spatial relations in probabilistic representations. Viewpoint maps are generated based on the knowledge of the road structure and driving patterns, which provide a prediction of the viewpoints of a vehicle whenever it happens at a certain location. Extensive experiments are conducted using an onboard camera on multilane motor ways in Beijing. A large-scale data set that contains more than 30 000 labeled ground truths for both fully and partially observed vehicles in different viewpoints across various traffic density scenes is developed. The data set will be opened to the society together with this publication.

Real-time Dense Disparity Estimation based on Multi-Path Viterbi for Intelligent Vehicle Applications.

This paper proposes a new real-time stereo matching algorithm paired with an online auto-rectification framework. The algorithm treats disparities of stereo images as hidden states and conducts Viterbi process at 4 bi-directional paths to estimate them. Structural similarity, total variation constraint, and a specific hierarchical merging strategy are combined with the Viterbi process to improve the robustness and accuracy. Based on the results of Viterbi, a convex optimization equation is derived to estimate epipolar line distortion. The estimated distortion information is used for the online compensation of Viterbi process at an auto-rectification framework. Extensive experiments were conducted to compare proposed algorithm with other practical state-of-the-art methods for intelligent vehicle applications.

Robust road boundary estimation for intelligent vehicles in challenging scenarios based on a semantic graph

This paper presents a stereovision-based detection and tracking approach of the drivable road boundary, designed for navigating an intelligent vehicle through challenging traffic scenarios, and increment road safety in such scenarios with advanced driver assistance systems (ADAS). It is based on a formulation of stereo with homography associated with a semantic graph constructed from the traffic scene. Under this formulation, we employ the Viterbi algorithm and propose a sophisticated measure of the probability of the state sequence in the semantic graph to find the most likely boundary between the road and non-road regions. The results are then refined by a post-processing step with the RANdom Sample Consensus (RANSAC) algorithm to obtain the locations and curvatures of the lateral road boundaries. Experimental results on a wide variety of typical but challenging real road scenes have substantiated the effectiveness as well as robustness of the proposed approach.

Drivable road region detection based on homography estimation with road appearance and driving state models

Road detection is one of the key issues for autonomous driving. In this paper, we present a drivable road region detection method based on homography estimation with road appearance and driving state models. In the method, the planar road region is detected and objects inside the region are localized through a 2D projective transformation between the stereo image pair by computing the homography induced by the road plane dynamically. This method is mainly composed of three modules: 1) preliminary classification module, which selects the most appropriate classifier from the road appearance model to detect the preliminary road-like region; 2) feature-based detection module, which finds the correspondences of feature points on the road plane to estimate the homography for the first image pair, and then extracts the drivable road region; 3) area-based detection module, a nonlinear optimization process, uses the results obtained in module 2 as the initial values for the homography estimation as well as drivable road region detection of the subsequent image pairs with the driving state model based on sequential information. The combination of these three modules uses both image evidence and temporal information; meanwhile, an error correction mechanism is applied. Therefore, more accurate as well as robust estimation of the homography can be expected, and so is the drivable road region detection. Experimental results on real road scenes have substantiated the effectiveness as well as robustness of the proposed method.