Lucas Malta

International Large-Scale Vehicle Corpora for Research on Driver Behavior on the Road

This paper considers a comprehensive and collaborative project to collect large amounts of driving data on the road for use in a wide range of areas of vehicle-related research centered on driving behavior. Unlike previous data collection efforts, the corpora collected here contain both human and vehicle sensor data, together with rich and continuous transcriptions. While most efforts on in-vehicle research are generally focused within individual countries, this effort links a collaborative team from three diverse regions (i.e., Asia, American, and Europe). Details relating to the data collection paradigm, such as sensors, driver information, routes, and transcription protocols, are discussed, and a preliminary analysis of the data across the three data collection sites from the U.S. (Dallas), Japan (Nagoya), and Turkey (Istanbul) is provided. The usability of the corpora has been experimentally verified with a Cohens kappa coefficient of 0.74 for transcription reliability, as well as being successfully exploited for several in-vehicle applications. Most importantly, the corpora are publicly available for research use and represent one of the first multination efforts to share resources and understand driver characteristics. Future work on distributing the corpora to the wider research community is also discussed.

A Study of Driver Behavior Under Potential Threats in Vehicle Traffic

Although, in recent years, significant developments have been made in road safety, traffic statistics indicate that we still need significant improvements in the field. Since traffic accidents usually reflect human factors, in this paper, we focus on clarifying the understanding of driver behaviors under hazardous scenarios. Brake pedal signals or driver speech, or both, are utilized to detect incidents from a real-world driving database of 373 drivers. Results are then analyzed to address the individuality in driver behaviors, the multimodality of driver reactions, and the detection of potentially dangerous locations. All of the existing 25 potentially hazardous scenes in the database are hand labeled and categorized. Based on the joint histograms of behavioral signals and their time derivatives, a detection feature is proposed and satisfactorily applied to the indication of anomalies in driving behavior. Seventeen scenes, where a reaction utilizing the brake pedal was observed, are detected with a true positive (TP) rate of 100% and a false positive (FP) rate of 4.1%. We demonstrate the relevance of considering behavior individuality. During 11 scenes, the drivers verbally reacted. Scenes that included high-energy words are adequately detected by the speech-based method, which achieved a TP rate of 54% for an FP rate of 6.4%. The integration of different behavior modalities satisfactorily boosts the detection of the most subjectively hazardous situations, which suggests the importance of considering multimodal reactions. Finally, a strong relationship is presented between locations where potentially hazardous situations occurred and areas of frequent strong braking.

Analysis of Real-World Driver's Frustration

This paper investigates a method for estimating a drivers spontaneous frustration in the real world. In line with a specific definition of emotion, the proposed method integrates information about the environment, the drivers emotional state, and the drivers responses in a single model. Driving data are recorded using an instrumented vehicle on which multiple sensors are mounted. While driving, drivers also interact with an automatic speech recognition (ASR) system to retrieve and play music. Using a Bayesian network, we combine knowledge on the driving environment assessed through data annotation, speech recognition errors, the drivers emotional state (frustration), and the drivers responses measured through facial expressions, physiological condition, and gas- and brake-pedal actuation. Experiments are performed with data from 20 drivers. We discuss the relevance of the proposed model and features of frustration estimation. When all of the available information is used, the overall estimation achieves a true positive rate of 80% and a false positive rate of 9% (i.e., the system correctly estimates 80% of the frustration and, when drivers are not frustrated, makes mistakes 9% of the time).

Multi-modal real-world driving data collection, transcription, and integration using Bayesian Network

In this paper we present our on-going data collection of multi-modal real-world driving. Video, speech, driving behavior, and physiological signals from 150 drivers have already been collected. To provide a more meaningful description of the collected data, we propose a transcription protocol based on six major groups: driver mental state, driver actions, driverpsilas secondary task, driving environment, vehicle status, and speech/background noise. Data from 30 drivers are transcribed. We then show how transcription reliability can be improved by properly training annotators. Finally, we integrate transcriptions, driving behavior, and physiological signals using a Bayesian network for estimating a driverpsilas level of irritation. Estimations are compared to actual values, assessed by the drivers themselves. Preliminary results are very encouraging.

Multimodal estimation of a driver's spontaneous irritation

In this paper we present our latest achievements in the continuous estimation of a drivers spontaneous irritation. Experiments are conducted with data from 20 drivers, recorded under real driving conditions. While driving, participants also interact with a speech dialogue system to retrieve and play music. A fusion method is proposed to integrate information on the driving environment, driver behavior, drivers physiological state, and speech recognition results. Overall, we are able to correctly detect 80% (true positive rate) of the irritation, and, when drivers are not irritated, we only make mistakes 9% of the time (false positive rate). Results also support the relevance of gas- and brake-pedal operation as well as speech recognition results in irritation estimation.

A multimedia corpus of driving behaviors

In this paper we present our multimedia corpus of real-world driving data (NUDrive), built with the primary objective of firming foundations for applying digital signal processing technologies in the vehicular environment. NUDrive is a content rich corpus composed of driving, speech, video, and physiological signals. So far, we have collected data from 250 drivers, who drove an instrumented vehicle under very similar conditions. In order to provide a more meaningful description of the situations drivers experience, a comprehensive data annotation protocol is proposed. We also briefly present a multimedia processing system, which uses information from various sources in NUDrive to implement a context-dependent estimation of a drivers spontaneous frustration. Results are encouraging and stress the relevance of content rich driving corpora to driver behavior modeling.

Towards the detection of potentially hazardous situations in vehicle traffic using driver speech and brake pedal operation

A method for automatic detection of potentially dangerous situations in motor vehicle traffic is introduced. Unlike precedent works, which typically relied on camera arrays or road‐traffic monitoring sensors to detect collision incidents, the proposed approach specifically incorporates changes in a drivers’ behavior, detected through driver speech and brake pedal operation. Experiments were performed using a large real‐world multimedia driving database of 493 drivers, obtained from the Centre for Integrated Acoustic Information Research (CIAIR, Nagoya University). The drivers, who interacted verbally with a human operator, uttered expletive words to express negative feelings in 11 of the 25 situations that we selected as potentially hazardous. In 17 of them, sudden and intense compression of the brake pedal was observed. The proposed lexicographical speech‐feature‐based method also detected 33 false alarms to detect 80% of these 11 scenes. As for the other 17 scenes, our method based on two‐dimensional hi...