Joel C. McCall

Looking-In and Looking-Out of a Vehicle: Computer-Vision-Based Enhanced Vehicle Safety

This paper presents investigations into the role of computer-vision technology in developing safer automobiles. We consider vision systems, which cannot only look out of the vehicle to detect and track roads and avoid hitting obstacles or pedestrians but simultaneously look inside the vehicle to monitor the attentiveness of the driver and even predict her intentions. In this paper, a systems-oriented framework for developing computer-vision technology for safer automobiles is presented. We will consider three main components of the system: environment, vehicle, and driver. We will discuss various issues and ideas for developing models for these main components as well as activities associated with the complex task of safe driving. This paper includes a discussion of novel sensory systems and algorithms for capturing not only the dynamic surround information of the vehicle but also the state, intent, and activity patterns of drivers

Driver Behavior and Situation Aware Brake Assistance for Intelligent Vehicles

This paper deals with the development of Human-Centric Intelligent Driver Assistance Systems. Rear-end collisions account for a large portion of traffic accidents. To help mitigate this problem, predictive braking systems and adaptive cruise control systems have been developed. However, these types of systems usually rely solely on the vehicle and vehicle surround sensors, either ignoring the human component of driving or learning the drivers control behavior using only these sensors. As with all human-computer interfaces, this has the potential to work against the driver, distract the driver further, or even annoy the driver so that the driver ignores or disables the system. It is, therefore, important to directly take the drivers intended actions into account when designing a driver assistance system. By using a probabilistic model for the system, warnings and preventative measures can be constructed based on varying levels of situational severity and driver attentiveness and intent. The research is based upon carefully conducted experimental trials involving a human subjects driving in natural manner and on typical freeways in the USA. The experiments, designed by inputs from cognitive scientist, were conducted in a specially designed instrumented vehicle to record important cues associated with drivers behavior, vehicle state, and vehicle surround in a synchronized manner. Quantitative results and analysis of the experimental trials are presented to show the feasibility and promise of this framework to predict the drivers intent to brake, the need for braking given the current situation, and at what level the driver should be warned

An integrated, robust approach to lane marking detection and lane tracking

Lane Detection is a difficult problem because of the varying road conditions that one can encounter while driving. We propose a method for lane detection using steerable filters. Steerable filters provide robustness to lighting changes and shadows and perform well in picking out both circular reflector road markings as well as painted line road markings. The filter results are then processed to eliminate outliers based on the expected road geometry and used to update a road and vehicular model along with data taken internally from the vehicles. Results are shown for a 9000-frame image sequence that include varying lane markings, lighting conditions, showing, and occlusion by other vehicles.

Lane Change Intent Analysis Using Robust Operators and Sparse Bayesian Learning

In this paper, we demonstrate a driver intent inference system that is based on lane positional information, vehicle parameters, and driver head motion. We present robust computer vision methods for identifying and tracking freeway lanes and driver head motion. These algorithms are then applied and evaluated on real-world data that are collected in a modular intelligent vehicle test bed. Analysis of the data for lane change intent is performed using a sparse Bayesian learning methodology. Finally, the system as a whole is evaluated using a novel metric and real-world data of vehicle parameters, lane position, and driver head motion.

Human Behavior Based Predictive Brake Assistance

Driver assistance systems have both the potential to alert the driver to critical situations and distract or annoy the driver if the driver is already aware of the situation. As systems attempt to preemptively warn drivers more and more in advance, this problem becomes exacerbated. We present a predictive braking assistance system that identifies not only the need for braking action, but also whether or not a braking action is being planned by the driver. Our system uses a Bayesian framework to determine the criticality of the situation by assessing (1) the probability that braking should be performed given observations of the vehicle and surround and (2) the probability that the driver intends to perform a braking action. We train and evaluate our system using over 22 hours of data collected from real driving scenarios with 28 different drivers

Design of an instrumented vehicle test bed for developing a human centered driver support system

We introduce a new type of intelligent vehicle test-bed that is enabling new research in the field. This new test-bed is designed to capture not just a portion of the vehicle surround, but rather the entire vehicle surround as well as the vehicle interior and vehicle state for extended periods of time. This is accomplished using multiple modalities of sensor systems so that it conform a complete context of the vehicle. This allows new research to be performed in intelligent vehicle algorithm development and allows studies in driver behavior to be performed. We also show results from some of the research being performed using this test-bed.

Performance evaluation of a vision based lane tracker designed for driver assistance systems

Driver assistance systems that monitor driver intent, warn drivers of lane departures, or assist in vehicle guidance are all being actively research and even put into commercial production. It is therefore important to take a critical look at key aspects of these systems, one of which being lane position tracking. In this paper we present an analysis of lane position tracking in the context of driver support systems and examine previous research in this area. Using this analysis we present a lane tracking system designed to work well under a variety of road and environmental conditions. We examine what types of metrics are important for evaluating lane position accuracy for specific overall system objectives. A detailed quantitative evaluation of the system is presented in this paper using a variety of metrics and test conditions.

Visual context capture and analysis for driver attention monitoring

Driver distraction is recognized as a major factor in the cause of automobile accidents. Therefore, it is extremely important for an intelligent driver support system to be able to monitor the drivers attentive state. This paper proposes a system to monitor driver attention based on a variety of information sources. The LISA-Q test vehicle is used to synchronously capture video, audio, vehicle information, LASER RADAR information, and GPS information as input to the driver state evaluation. Information about the drivers facial affects, lane keeping, steering movements, and time headway are all extracted from the multimodal data streams and evaluated.

A collaborative approach for human-centered driver assistance systems

This paper describes an interdisciplinary research collaboration to design a human-centered driver assistance system. Driving behavior is captured using a novel intelligent vehicle test bed. The synchronized capture of driver behavior and driving context provides an empirical basis for the design and evaluation.

Pose invariant affect analysis using thin-plate splines

This paper introduces a method for pose-invariant facial affect analysis and a real-time system for facial affect analysis using this method. The method is centered on developing a feature vector that is more robust to rigid body movements while retaining information important to facial affect analysis. This feature vector is produced using thin-plate splines to extract affine transformations independently from nonlinear transformations quickly and efficiently. The affine portion can be used to describe the rigid body motion because planar motions in a perspective projection can be approximated by an affine transformation. Removing the affine portion and using the nonlinear portion of the thin-plate spline warping provides information on the nonlinear motion caused by facial affects. The real-time system developed using this method is composed of three main components: facial landmark tracking, feature vector extraction, and affect classification. The system processes streaming video in real-time. Testing was performed to examine the invariance to rotation as well as subject independence of the system. Finally, its application in real-world environments is discussed.