Xuetao Zhang

Interactive Road Situation Analysis for Driver Assistance and Safety Warning Systems: Framework and Algorithms

Road situation analysis in Interactive Intelligent Driver-Assistance and Safety Warning (I2DASW) systems involves estimation and prediction of the position and size of various on-road obstacles. Real-time processing, given incomplete and uncertain information, is a challenge for current object detection and tracking technologies. This paper proposed a development framework and novel algorithms for road situation analysis based on driving action behavior, where the safety situation is analyzed by simulating real driving action behaviors. First, we review recent development and trends in road situation analysis to provide perspective for the related research. Second, we introduce a road situation analysis framework, where onboard sensors provide information about drivers, traffic environment, and vehicles. Finally, on the basis of the previous frameworks, we proposed multiple-obstacle detection and tracking algorithms using multiple sensors including radar, lidar, and a camera, where a decentralized track-to-track fusion approach is introduced to fuse these sensors. In order to reduce the effect of obstacle shape and appearance, we cluster lidar data and then classify obstacles into two categories: static and moving objects. Future collisions are assessed by computation of local tracks of moving obstacles using extended Kalman filter, maximum likelihood estimation to fuse distributed local tracks into global tracks, and finally, computation of future collision distribution from the global tracks. Our experimental results show that our approach is efficient for road situation evaluation and prediction

A SLAM algorithm based on the central difference Kalman filter

This paper presents an central difference Kalman filter (CDKF) based Simultaneous Localization and Mapping (SLAM) algorithm, which is an alternative to the classical extended Kalman filter based SLAM solution (EKF-SLAM). EKF-SLAM suffers from two important problems, which are the calculation of Jacobians and the linear approximations to the nonlinear models. They can lead the filter to be inconsistent. To overcome the serious drawbacks of the previous frameworks, Sterlings polynomial interpolation method is employed to approximate nonlinear models. Combined with the Kanlman filter framework, CDKF is proposed to solve the probabilistic state-space SLAM problem. The proposed approach improves the filter consistency and state estimation accuracy. Both simulated experiments and bench mark data set are used to demonstrating the superiority of the proposed algorithm.

Vehicle detection using an extended Hidden Random Field model

Prevent collision with other vehicles is crucial for developing advanced driver assistance systems. Vision-based approaches for vehicle detection attract more attention than those using other sensors. In this study, we address the problem of detecting front vehicles in still images. Unlike traditional methods which mainly based on the holistic appearance of vehicles, we adopted a local part based model. We extended the Hidden Random Field (HRF) model to incorporate logistic regression classifiers into unary potentials. The proposed model was trained and tested on a set of real images captured by an on-board camera. The results showed that the effectiveness of the approach, and a better performance could be found when the vehicle was occluded by other vehicles.

Head pose estimation using isophote features for driver assistance systems

In this paper, we consider the problem of estimating the pose of a driver from video data. We propose to use isophote features to improve classification performance when illumination varies. In particular, we both use the direction and curvature features. The features are encoded into direction histogram and curvature histogram in the blocks of the image. Experimental results show that the proposed features could well describe the structure of driver face image, and have a good performance in real scene.

Vehicle detection under varying poses using Conditional Random Fields

Traditional vision based vehicle detection methods are more successful in detecting front and rear vehicles. However, the problem of detecting vehicles under various poses still presents a great deal of difficulty. Pose variation leads to limit the use of vision based driver assistance systems. In this paper, we present a Conditional Random Fields (CRFs) based algorithm that can detect vehicles under various poses. We treat this problem in a different way. We extract textural properties from small image patches as well as colors. Then CRFs model is employed to incorporate the contextual information. Firstly, we classify these patches into vehicular surfaces or background surfaces. Then we use clustering algorithm to eliminate the false alarms and detect multiple vehicles. From the quantitative evaluation of the proposed methods, our algorithm can be used in many practical applications that do not need accurate segmentation of vehicles.

Unscented SLAM with conditional iterations

As reported, the extended Kalman Filter based Simultaneous Localization and Mapping (SLAM) algorithm has two serious drawbacks, namely the linear approximation of non-linear functions and the calculation of Jacobian matrices. These can introduce estimation error and induce a great ambiguity for data association. For overcoming these drawbacks, this paper presents an improved SLAM solution, based on the Unscented Kalman Filter (UKF) with conditional iterations (UiSLAM). Since the UKF can improve the performance of filters, it can be used to overcome the drawbacks of the previous frameworks. When the loop is closed, the condition to perform iterated update is satisfied. Then the iterative update procedure employed in the iterated extended Kalman Filter (IEKF) is implemented. This approach combines the virtues of IEKF and UKF for solving the SLAM problems and improves accuracy of the state estimation. Both the simulation and experimental results are proposed to illustrate the superiority of the UiSLAM algorithm over previous approaches.

Visual recognition of driver hand-held cell phone use based on hidden CRF

In this paper, we propose an automatic system that recognizes drivers abnormal behavior, i.e. cell phone use. Drivers actions are captured using a camera mounted above the dash board. Then the observed features are input into a Hidden Conditional Random Fields (HCRF) model. To incorporate long range dependencies, features are collected within a local window from neighbor sites. We evaluate the presented algorithm on the real video segments, and the results show that the system can successfully recognize the behavior.

Visual Appearance-Based Unmanned Vehicle Sequential Localization

Localizationis of vital importance for an unmanned vehicle to drive on the road. Most of the existing algorithms are based on laser range finders, inertial equipment, artificial landmarks, distributing sensors or global positioning system(GPS) information. Currently, the problem of localization with vision information is most concerned. However, vision-based localization techniquesare still unavailable for practical applications. In this paper, we present a vision-based sequential probability localization method. This method uses the surface information of the roadside to locate the vehicle, especially in the situation where GPS information is unavailable. It is composed of two step, first, in a recording stage, we construct a ground truthmap with the appearance of the roadside environment. Then in an on-line stage, we use a sequential matching approach to localize the vehicle. In the experiment, we use two independent cameras to observe the environment, one is left-orientated and the other is right. SIFT features and Daisy features are used to represent for the visual appearance of the environment. The experiment results show that the proposed method could locate the vehicle in a complicated, large environment with high reliability.

Vehicle active safety applications by fusing multiple-sensor for the Springrobot system

This paper presents the Springrobot system and its applications in vehicle active safety, and it is divided into three parts. Firstly, we describe in detail Springrobot intelligent vehicles hardware structure, internal and external sensing systems, sensor fusion, and its control system. The second part is its vehicle active safety applications including lane departure warning system, forward collision avoidance assistance system, and onroad vehicle detection system using BGF. We describe not only the system structure and algorithm, but also the experimental results. The third part is that we conclude with related research field that we are currently pursuing and direction for future work in, for example, obstacle detection and classification using statistical learning methods, vehicular telematics.

A Denoising Scheme for Randomly Clustered Noise Removal in ICCD Sensing Image

An Intensified Charge-Coupled Device (ICCD) image is captured by the ICCD image sensor in extremely low-light conditions. Its noise has two distinctive characteristics. (a) Different from the independent identically distributed (i.i.d.) noise in natural image, the noise in the ICCD sensing image is spatially clustered, which induces unexpected structure information; (b) The pattern of the clustered noise is formed randomly. In this paper, we propose a denoising scheme to remove the randomly clustered noise in the ICCD sensing image. First, we decompose the image into non-overlapped patches and classify them into flat patches and structure patches according to if real structure information is included. Then, two denoising algorithms are designed for them, respectively. For each flat patch, we simulate multiple similar patches for it in pseudo-time domain and remove its noise by averaging all the simulated patches, considering that the structure information induced by the noise varies randomly over time. For each structure patch, we design a structure-preserved sparse coding algorithm to reconstruct the real structure information. It reconstructs each patch by describing it as a weighted summation of its neighboring patches and incorporating the weights into the sparse representation of the current patch. Based on all the reconstructed patches, we generate a reconstructed image. After that, we repeat the whole process by changing relevant parameters, considering that blocking artifacts exist in a single reconstructed image. Finally, we obtain the reconstructed image by merging all the generated images into one. Experiments are conducted on an ICCD sensing image dataset, which verifies its subjective performance in removing the randomly clustered noise and preserving the real structure information in the ICCD sensing image.