Zhaozheng Hu

Grid-based pavement crack analysis using deep learning

Pavement crack detected plays an important role in pavement maintenance. Image recognition is a traditional way for pavement crack detected. Recently, deep learning is a state-of-the-art method for target detection. CNN (convolutional neural network), a significant method in deep learning, is widely used in image target detection and brings about breakthroughs. However, CNN has not been applied to pavement crack detection. In this paper, we apply CNN to detect pavement crack and PCA (Principal Component Analysis) to classify the detected pavement cracks. Firstly, two databases are obtained by using two different scales of grid (32×32, 64×64) to segment pavement images. Each database has 30000 images for training set. We obtain two kinds of trained CNN. Each CNN is trained by one training set, which is part of each scale databases. We use trained CNN to detect the existence of pavement crack in corresponding scale grids. We confirm the scale of segment grid by comparing the results of pavement crack detected. Secondly, we only keep the grids containing crack and achieve the skeleton of crack in a pavement image. Lastly, we use PCA to analyse the skeleton of crack. The classification of crack can be obtained. The F-measure for crack detection is 94.7%. Meanwhile, the proposed method achieves 97.2%, 97.6% and 90.1% correct rate of classification for longitudinal crack, transverse crack and alligator crack, respectively. The results show proposed method can detect the pavement crack and evaluate the type of crack precisely.

Image Sequence Matching Using Both Holistic and Local Features for Loop Closure Detection

Simultaneous localization and mapping (SLAM) has a wide range of applications, such as mobile robots, intelligent vehicle localization, and intelligent transportation system. However, loop closure detection is a challenge task for SLAM. This task concerns the difficulty of recognizing already mapped areas. To this end, this paper proposes a novel loop closure detection method called image sequence matching (ISM), which only uses a low-cost monocular camera. This method first divides the already mapped areas into some “feature-zones.” One feature-zone is selected by a novel topological detection model. Then, we adopt two different feature spaces to make sequence matching between query image and feature-zone. Last but not least, we propose a novel clustering method called voting K-nearest neighbor to fuse candidates. As a result, the ISM method has been validated by using collection data sets and public data sets, which were collected along different routes, covering different times and weather conditions. The total lengths of these routes are more than 10 km. Experimental results show that the ISM method can adapt to different times with good detection stability in varying scenarios. The mean of detection errors is all less than 1 frame and the detection accuracies are all more than 90% in these scenarios. Compared with other methods, the proposed method has high accuracy and great robustness.

Integration of GPS, Monocular Vision, and High Definition (HD) Map for Accurate Vehicle Localization

Self-localization is a crucial task for intelligent vehicles. Existing localization methods usually require high-cost IMU (Inertial Measurement Unit) or expensive LiDAR sensors (e.g., Velodyne HDL-64E). In this paper, we propose a low-cost yet accurate localization solution by using a custom-level GPS receiver and a low-cost camera with the support of HD map. Unlike existing HD map-based methods, which usually requires unique landmarks within the sensed range, the proposed method utilizes common lane lines for vehicle localization by using Kalman filter to fuse the GPS, monocular vision, and HD map for more accurate vehicle localization. In the Kalman filter framework, the observations consist of two parts. One is the raw GPS coordinate. The other is the lateral distance between the vehicle and the lane, which is computed from the monocular camera. The HD map plays the role of providing reference position information and correlating the local lateral distance from the vision and the GPS coordinates so as to formulate a linear Kalman filter. In the prediction step, we propose using a data-driven motion model rather than a Kinematic model, which is more adaptive and flexible. The proposed method has been tested with both simulation data and real data collected in the field. The results demonstrate that the localization errors from the proposed method are less than half or even one-third of the original GPS positioning errors by using low cost sensors with HD map support. Experimental results also demonstrate that the integration of the proposed method into existing ones can greatly enhance the localization results.

Wi-Fi and vision integrated localization for reverse vehicle-searching in underground parking lot

With the expansion of the underground parking lot, reverse vehicle-searching has become urgent problems that need to be solved. In this paper, we proposed a Wi-Fi and vision integrated reverse vehicle-searching method. We transform the reverse vehicle-searching into indoor self-positioning. And we proposed a multi-scale indoor positioning method which includes coarse localization with Wi-Fi matching and image-level localization with ORB based visual matching. Once the position of the host of the vehicle is located, we exploit A∗ algorithm to plan the route from the host to the position of the vehicle that has been recorded in advance. The proposed method had been tested in two different datasets of 6,000 square meters and 12,000 square meters underground parking lot. From the results, the proposed Wi-Fi and vision integrated method can achieve at least 97% reverse vehicle-searching success rate. In both test scenarios, the localization errors are sub-meter in average.

Robust road lane detection from shape and color feature fusion for vehicle self-localization

Vehicle self-localization plays critical role on safety driving assistant and automatic driving. Vehicle self-localization can be applied to Lane Departure Warning Systems (LDWS) in order to provide higher vehicle safety and is very useful in driving behavior research. This paper presents a lane detection method by integrating lane shape and color features as well as using the detected lane for online vehicle position calculation. First, a statistical color model, called Gaussian Statistical Color Model (G-SCM) is implemented for image processing to extract the region of interest and to eliminate the interference of the lane. Moreover, it is to detect the lane and get the vanishing point in the direction of the lane by utilizing the improved Hough transform on the selected image area. In the last step, we can calculate the vehicle position (yaw angle of vehicle to the lane and the distance between the vehicle and the detected lane) by the corresponding relation of the image coordinate system and the world coordinate system. Experimental results show that this method can quickly and accurately detect the lane and complete a remarkable vehicle position.

Online estimation for vehicle center of gravity height based on unscented Kalman filter

The Center of Gravity height of heavy-duty vehicles will drift apparently when the load changes, while online estimation of the vehicle CG height is of high importance to the vehicle active safety system. An unscented Kalman filter based on three-degree-of-freedom (3-DOF) vehicle dynamics model is proposed to acquire the real-time value of vehicle CG height through sensing vehicle speed, front and rear wheel speed etc. The results of combined simulation studies based on TruckSim and MATLAB/Simulink show that the estimation algorithm is able to obtain the true value of CG height in a short time with a steady average error rate less than 12%. The results are instructive for the vehicle dynamic control system.

Fast self-localization of autonomous vehicles from visual map

Self-localization is a fundamental task for connected autonomous vehicles. This paper proposes a visual self-localization method by matching images from an in-vehicle monocular camera with those from a visual map. We use the classic ORB (Oriented FAST and Rotated BRIEF) method to encode both holistic and local features of an input image. In this method, each input image is normalized into a 63 × 63 image patch. The patch center is set as the ORB point position and the corresponding ORB descriptor is used as holistic feature. Besides, we also extract local features by representing all the ORB descriptors extracted from the original input image with visual words by using the classic Bag-of-Words (BOW) method. Finally, we extend the use of hybrid K-nearest neighbor (H-KNN) to fuse ORB-encoded holistic and local features for position or site recognition. The proposed self-localization method was validated by using actual images collected along a road segment of 3.2 km in Wuhan City, China, covering different road scenes, such as bridges, curved roads, straight roads, cross-roads, tunnels, etc. Experiment results show that the proposed method achieved 77.2% image recognition rate, with about 19ms in average for localization from one image. The average positioning errors were within 5m. The results demonstrate that the proposed method is promising in term of positioning accuracy and speed to develop low-cost self-localization device for autonomous vehicles.

A novel rollover warning method for ground vehicles based on smartphone built in GIS/GPS

In recent years, it has been shown that damages of casualties and property losses caused by vehicle rollover accidents are severe. Vehicle rollovers can easily cause secondary accidents such as chain collisions on freeways. Ground vehicles with relatively higher center of gravity on sharp curves are prone to rollover accidents. Therefore, it is of vital significance to design a rollover warning system for this type of vehicles. After summarizing current research achievements about vehicle rollover warning methods, the paper presents a novel rollover warning method for ground vehicles based on GIS/GPS. It firstly elaborates the method for road curve identification and curve radius estimation. Then, a curve speed model is built based on vehicle dynamics and road environment conditions. Lastly, it presents the architecture and field test verification of the rollover warning system. The main contributions of this paper are organized as follows: 1) Curve speed model is built by analyzing the correlation between the front wheel angle and the curve radius. Moreover, rollover limiting condition is analyzed based on vehicle roll dynamics modeling; 2) Curve road recognition based on GIS/GPS using smartphone platform. It presents a curve recognition algorithm based on acquiring road and vehicle parameters such as curve entry, driving direction and vehicle lateral acceleration; 3) Curve radius estimation based on road curve fitting and radius calculation. After obtaining location of each circular section on the curve through the digital map, the center of each circular section is determined and then the average radius of the curve is calculated.

Registration of image and 3D LIDAR data from extrinsic calibration

Camera and laser rangefinder (LRF) are widely used in various mobilized systems, such as intelligent vehicle, autonomous robot, etc. And extrinsic calibration is essential and basically the first step to integrate image and 3D LIDAR data. This paper presents a flexible method for extrinsic calibration of a pin-hole camera and a 3D Laser Rangefinder (LRF). The method extends the chessboard pattern, which is originally used for camera calibration, for camera and 2D LRF calibration. It requires at least 3 input planes to determine the rotation and translation between these two sensors. The proposed method formulates the extrinsic calibration problem as to register point correspondences in the dual 3D space instead of plane registration in 3D space. And the rotation matrix and the translation vector are estimated separately. The extrinsic calibration results allow the registration of image and 3D LIDAR data by mapping all the LIDAR data onto the imaging plane. The proposed method has been tested with both simulation and real data. The experimental results show that the proposed algorithm is both accurate and practical.