Alexander Zelinsky

Driver Inattention Detection based on Eye Gaze-Road Event Correlation

Current road safety initiatives are approaching the limit of their effectiveness in developed countries. A paradigm shift is needed to address the preventable deaths of thousands on our roads. Previous systems have focused on one or two aspects of driving: environmental sensing, vehicle dynamics or driver monitoring. Our approach is to consider the driver and the vehicle as part of a combined system, operating within the road environment. A driver assistance system is implemented that is not only responsive to the road environment and the drivers actions but also designed to correlate the drivers eye gaze with road events to determine the drivers observations. Driver observation monitoring enables an immediate in-vehicle system able to detect and act on driver inattentiveness, providing the precious seconds for an inattentive human driver to react. We present a prototype system capable of estimating the drivers observations and detecting driver inattentiveness. Due to the “look but not see” case it is not possible to prove that a road event has been observed by the driver. We show, however, that it is possible to detect missed road events and warn the driver appropriately.

Correlating driver gaze with the road scene for driver assistance systems

A driver assistance system (DAS) should support the driver by monitoring road and vehicle events and presenting relevant and timely information to the driver. It is impossible to know what a driver is thinking, but we can monitor the drivers gaze direction and compare it with the position of information in the drivers viewfield to make inferences. In this way, not only do we monitor the drivers actions, we monitor the drivers observations as well. In this paper we present the automated detection and recognition of road signs, combined with the monitoring of the drivers response. We present a complete system that reads speed signs in real-time, compares the drivers gaze, and provides immediate feedback if it appears the sign has been missed by the driver.

Road scene monotony detection in a fatigue management driver assistance system

Automated fatigue detection devices show much promise in combating fatigue related accidents. One aspect which hampers the introduction of these technologies is context awareness. In this paper we develop and evaluate a road scene monotony detector. The detector can be used to give context awareness to fatigue detection tools to minimise false positives. The approach could also be used by road makers to quantify monotony on fatigue prone stretches of road. The detector uses MPEG compression to measure the change in information content of the road scene over time. We show that the detector correlates highly with human identified monotonous scenes. The technique is consistent over time and applicable for day and night operation. The compression is augmented with lane tracking data to distinguish between otherwise difficult cases. The detector is integrated into a fatigue management driver assistance system.

MAP ZDF segmentation and tracking using active stereo vision: Hand tracking case study

A maximum a posterior probability zero disparity filter (MAP ZDF) ensures coordinated stereo fixation upon an arbitrarily moving, rotating, re-configuring hand, performing marker-less pixel-wise segmentation of the hand. Active stereo fixation permits real-time foveal hand tracking and segmentation over a large visual workspace, allowing investigation of unrestricted natural human gesturing. Hand segmentation is shown to be robust to lighting conditions, defocus, hand colour variation, foreground and background clutter including non-tracked hands, and partial or gross occlusions including those due to non-tracked hands. The system operates at approximately 27fps on a 3GHz single processor PC.

Humanoid Vision Resembles Primate Archetype

Perception in the visual cortex and dorsal stream of the primate brain includes important visual competencies, such as: a consistent representation of visual space despite eye movement; egocentric spatial perception; attentional gaze deployment; and, coordinated stereo fixation upon dynamic objects. These competencies have emerged commensurate with observation of the real world, and constitute a vision system that is optimised, in some sense, for perception and interaction. We present a robotic vision system that incorporates these competencies. We hypothesise that similarities between the underlying robotic system model and that of the primate vision system will elicit accordingly similar gaze behaviours. Psychophysical trials were conducted to record human gaze behaviour when free-viewing a reproducible, dynamic, 3D scene. Identical trials were conducted with the robotic system. A statistical comparison of robotic and human gaze behaviour has shown that the two are remarkably similar. Enabling a humanoid to mimic the optimised gaze strategies of humans may be a significant step towards facilitating human-like perception.

Automotive Safety Solutions through Technology and Human-Factors Innovation

Advanced Driver Assistance Systems (ADAS) provide warnings and in some cases autonomous actions to increase driver and passenger safety by combining sensor technologies and situation awareness. In the last 10 years progressed from prototype demonstrators to full product deployment in motor vehicles. Early ADAS examples include Lane Departure Warning (LDW) and Forward Collision Warning (FCW) systems have been developed to warn drivers of potentially dangerous situations. More recently, driver inattention systems have made their debut. These systems are tackling one of the major causes of fatalities on roads – drowsiness and distraction. This paper describes DSS, a driver inattention warning system which has been developed by Seeing Machines for commercial applications, with an initial focus on heavy vehicle fleet management applications. A case study reporting a year-long real-world deployment of DSS is presented. The study showed the effectiveness of the DSS technology in mitigating driver inattention in a sustained manner.

Context Sensitive Driver Assistance Based on Gaze – Road Scene Correlation

Introducing a new approach to intelligent vehicle systems. Previous systems have focused on one or two aspects of: environmental sensing, vehicle dynamics or driver monitoring. Our approach is to consider the driver and the vehicle as part of a combined system, operating within the road environment. A driver assistance system is implemented that is not only responsive to the road environment and the driver’s actions but also designed to correlate the driver’s gaze with the road scene to determine the driver’s observations. Driver observation monitoring enables the system to anticipate the driver’s needs, enabling: context relevant information selection, redundant information suppression and a natural acknowledgement interface.

Bimodal Active Stereo Vision

We present a biologically inspired active vision system that incorporates two modes of perception. A peripheral mode provides a broad and coarse perception of where mass is in the scene in the vicinity of the current fixation point, and how that mass is moving. It involves fusion of actively acquired depth data into a 3D occupancy grid. A foveal mode then ensures coordinated stereo fixation upon mass/objects in the scene, and enables extraction of the mass/object using a maximum a-posterior probability zero disparity filter. Foveal processing is limited to the vicinity of the camera optical centres. Results for each mode and both modes operating in parallel are presented. The regime operates at approximately 15Hz on a 3GHz single processor PC.

Dependable autonomous systems

Today, we expect that all consumer technologies such as computers, ipods and motor vehicles all operate with the highest levels of performance, reliability and integrity. For robotics and automation technologies to successfully deployed into everyday situations, the general public quite rightly expect automation systems to be fully operational 100% of the time. People expect autonomous technologies to operate at higher levels of performance and safety than people themselves exhibit. For example smart car technologies are expected to cause ZERO accidents while human errors kill more 150,000 people on our roads world-wide every year!

Editorial Special Issue on the Fourth International Conference on Field and Service Robotics, 2003

much attention has been increasingly paid to the technology of field and service robotics. The FSR conferences have been held bi-a collection of selected papers from FSR03. To promote the robot industry, the potential needs and the market of robot technology have been investigated, and it has been predicted that field and service robots is the area most expected to enlarge the robot market rather than manufacturing. At FSR conferences, many exciting cutting-edge technologies have been introduced and discussed, and from this was proposed the probabilistic approach represented by SLAM (Simultaneous Localization and Mapping), which dramatically made solvable the localization and mapping issues in real environments. This technology is extensive, not only in robot science, but also in robot applications. Furthermore, the evolution in sensing technologies and infrastructures for robot navigation, such as laser range sensors, GPS, etc. have made great contribution to the localization, mapping, and navigation. Since the first FSR conference was held, great progress has been achieved in field and service robotics, and the available technology which has evolved has made feasible the introduction of mobile robots and vehicles to various actual applications. In addition, conventional research on mechanical design, perception, planning, and control as well as learning has also grown in these years. An intensive effort was made to accommodate to uncertainty and complexity in the environment, even in the human co-existent environment. Realization of the adaptive function of robots has been pursued. It should also be noted that attention has been increasingly paid to the services which can be delivered by robot technology rather than the function of the robot itself. Sixty-eight papers were submitted to FSR03, and 49 original papers were selected for presentation based on the outcome of a strict review process by the program committee members. The contents of the papers presented at FSR03 are very interesting. All the presented papers will be published as a monograph book of the STAR series from Springer, which is entitled Field and Service Robotics: Recent Advances in Research andApplications. For this special issue we selected 11 papers which were rated very highly by the reviewers. The authors of these invited papers were requested to submit revised and extended versions from those of the proceedings papers, and also to include some up-to-date results. All the papers were subjected to the IJRR normal review process, after which six papers were finally accepted for publication of …Since the first International Conference on Field and Service Robotics (FSR), which was held in Canberra, Australia, 1997, and organized by Prof. Alexander Zelinsky, much attention has been increasingly paid to the technology of field and service robotics. The FSR conferences have been held biannually since: FSR’99, Pittsburgh, USA, FSR’01, Helsinki, Finland, and FSR’03,Yamanashi, Japan. This special issue is a collection of selected papers from FSR’03. To promote the robot industry, the potential needs and the market of robot technology have been investigated, and it has been predicted that field and service robots is the area most expected to enlarge the robot market rather than manufacturing. At FSR conferences, many exciting cutting-edge technologies have been introduced and discussed, and from this was proposed the probabilistic approach represented by SLAM (Simultaneous Localization and Mapping), which dramatically made solvable the localization and mapping issues in real environments. This technology is extensive, not only in robot science, but also in robot applications. Furthermore, the evolution in sensing technologies and infrastructures for robot navigation, such as laser range sensors, GPS, etc. have made great contribution to the localization, mapping, and navigation. Since the first FSR conference was held, great progress has been achieved in field and service robotics, and the available technology which has evolved has made feasible the introduction of mobile robots and vehicles to various actual applications. In addition, conventional research on mechanical design, perception, planning, and control as well as learning has also grown in these years. An intensive effort was made to accommodate to uncertainty and complexity in the environment, even in the human co-existent environment. Realization of the adaptive function of robots has been pursued. It should also be noted that attention has been increasingly paid to the services which can be delivered by robot technology rather than the function of the robot itself. Sixty-eight papers were submitted to FSR‘03, and 49 original papers were selected for presentation based on the outcome of a strict review process by the program committee members. The contents of the papers presented at FSR’03 are very interesting. All the presented papers will be published as