James P. Bliss

Human probability matching behaviour in response to alarms of varying reliability

The goals of this research were to substantiate the existence of the cry-wolf effect for alarm responses, quantifying its effect on operator performance. A total of 138 undergraduate students performed two blocks of a cognitively demanding psychomotor primary task; at the same time, they were presented with alarms of varying reliabilities (25, 50 and 75% true alarms) and urgencies (green, yellow and red visual alarms presented concurrently with low-, medium- and high-urgency auditory civilian aircraft cockpit alarms). Alarm response frequencies were observed and analysed, and t-tests and repeated-measures MANOVAs were used to assess the effects of increasing alarm reliability on alarm response frequencies, speed and accuracy. The results indicate that most subjects (about 90%) do not respond to all alarms but match their response rates to the expected probability of true alarms (probability matching). About 10% of the subjects responded in the extreme, utilizing an all-or-none strategy. Implications of these results for alarm design instruction and further research are discussed.

The effectiveness of virtual reality for administering spatial navigation training to firefighters

Because fire rescue personnel often enter unfamiliar buildings to perform critical tasks like rescues, the importance of finding new and improved ways to train route navigation is becoming paramount. This research was designed to compare three methods for training firefighters to navigate a rescue route in an unfamiliar building. Thirty firefighters from the Madison County, Alabama, area were trained to navigate through the Administrative Science Building at The University of Alabama in Huntsville. The firefighters, who had not had any experience with the Administrative Science Building prior to the experiment, were randomly assigned to one of three experimental training groups: Blueprint, Virtual Reality, or No Training (Control). After training, we measured the total navigation time and number of wrong turns exhibited by firefighters in the actual building. Participants were required to rescue a mock baby (a life-sized doll) following the specific trained route. Measures of test performance were compared among groups by using analyses of variance (ANOVAs). The results indicated that firefighters trained with virtual reality or blueprints performed a quicker and more accurate rescue than those without training. Furthermore, the speed and accuracy of rescue performance did not differ significantly between virtual reality and blueprint training groups. These results indicate that virtual reality training, if constructed and implemented properly, may provide an effective alternative to current navigation training methods. The results are discussed with regard to theories of transfer of training and human performance in virtual environments.

Behavioural implications of alarm mistrust as a function of task workload

The research was conducted to investigate the effect of increasing primary task and alarm workload on alarm mistrust as reflected by alarm and primary task performances. A total of 126 undergraduate students performed a complex psychomotor task battery three times, with the number of concurrent tasks increasing each time. During their performance, the students were required to react to an alarm system (including visual and auditory components) of questionable reliability. Depending on the group to which participants were assigned, the alarm presentation rate constituted a low-, medium- or highworkload condition. Alarm response data (times, frequencies, accuracies) and primary task data (tracking error) were analyzed to assess performance differences as a function of primary and secondary task workload levels. Results generally supported the hypotheses: increasing primary task and alarm task workload degraded alarm response performance. Also, response frequencies supported earlier research suggesting that participants ‘probability match’ their response rates to alarm system reliability. The results are discussed with regard to the cry-wolf effect, attention theory and alarm system design.

The virtual environment performance assessment battery vepab: Development and evaluation1

The Virtual Environment Performance Assessment Battery (VEPAB) is a set of tasks developed to support research on training applications of virtual environment (VE) technology. VEPAB measures human performance on vision, locomotion, tracking, object manipulation, and reaction time tasks performed in three-dimensional, interactive VEs. It can be used to provide a general orientation for interacting in VEs and to determine both entry level performance and skill acquisition of users. In addition, VEPAB allows comparison of task performance, side effects and aftereffects, and subjective reactions across different VE systems. By providing benchmarks of human performance, VEPAB can promote continuity in training research involving different technologies, separate research facilities, and dissimilar subject populations. This paper describes the development of VEPAB and summarizes the results of two experiments, one to test the sensitivity of the tasks to differences between input control devices and the other to examine practice effects.

Investigation of Alarm-Related Accidents and Incidents in Aviation

With increases in technology, pilots have assumed the role of system monitors. As such, they have become dependent upon sensor-based alarm signals, some of which are unreliable. Researchers have shown that unreliable alarms foster mistrust and complacency by operators, and may degrade flight tasks and alarm reactions. To supplement available laboratory investigations, we tabulated the number of events involving true, false, and missed alarms within the Aviation Safety Reporting System database, the National Transportation Safety Board database, and the U.S. Armys Aviation Safety database. We also separated the results by aircraft type, avionics, flight phase, and specific problem. Our investigation revealed that alarm-related problems occur frequently across flight environments, and that false alarms remain a significant concern in aviation. Given the pervasiveness of alarm-related problems, it is important that future researchers devote considerable effort to improve aviation alarm systems.

REVERSAL OF THE CRY-WOLF EFFECT : AN INVESTIGATION OF TWO METHODS TO INCREASE ALARM RESPONSE RATES

In complex task environments, false alarms have been associated with less frequent and slower alarm responses. This research attempted to improve alarm responses using a hearsay method, in which participants were told that false alarms would be less frequent than they actually were, and an urgency method, in which the urgency of alarms was increased. Response frequency, speed, and accuracy of three groups of 20 students (Urgency, Hearsay, and Control) were compared across groups and sessions using analyses of variance and t tests. Both methods were successful; hearsay participants increased their response rates across sessions, and urgency participants decreased their response times. The results are discussed with regard to design of alarm systems and theory of human performance.

Distance Estimation in Virtual Environments

This paper describes an experiment to evaluate a procedure for measuring distance perception in immersive VEs. Forty-eight subjects viewed a VE with a Head Mounted Display (HMD), a Binocular Omni-Oriented Monitor (BOOM), or a computer monitor. Subjects estimated the distance to a figure of known height that was initially 40 ft away. As the figure moved forward, subjects indicated when the figure was perceived to be 30, 20, 10, 5, and 2.5 ft away. A separate group of 36 subjects performed the task in a real-world setting roughly comparable to the VE. VE distance estimation was highly variable across subjects. For distance perception involving a moving figure, in the VE conditions most subjects called out before the figure had closed to the specified distances. Distance estimation was least accurate with the monitor. In the real world, most subjects called out after the figure had closed to or passed the specified distances. Ways to improve the procedure are discussed.

Emergency signal failure: implications and recommendations

As automation becomes more prevalent in complex task systems, operators are confronted with a myriad of emergency signals. Although much research has been performed to improve the structure and function of signalling systems, it has concerned mainly those stimuli that are presumed to accurately indicate danger. The current report discusses the general area of emergency signal failure, its implications for complex task performance, and recommendations for design and research. A general description of the problem is followed by a review of the applicability of scientific theory to human behaviour during and following signal failure, and the authors conclude with recommendations for personnel training and warning signal design and research.

Effects of varying the threshold of alarm systems and workload on human performance

The purpose of this research was to investigate the effects of varying the threshold of alarm systems and workload on human response to alarm signals and performance on a complex task. A receiver operating characteristic (ROC) curve was selected to reflect the sensitivity of the alarm system. The threshold of the alarm system was manipulated by changing the value of beta along the ROC curve. A total of 84 students participated in experiment 1 and 48 students participated in experiment 2. Participants performed a compensatory-tracking, a resource management and a monitoring task. As expected, results showed that participants responded significantly faster to true alarm signals when they were using the system with the highest threshold under low-workload conditions. Results also indicated that changing the threshold of the alarm system had a significant effect on overall performance and this effect was greater under high-workload conditions. However, contrary to expectations, the highest level of performance was achieved by setting the threshold at a low level. Results from both experiments revealed that the advantage of faster alarm reaction time as a result of increasing the systems threshold was lost because of its increased probability of missed events.

The efficacy of a medical virtual reality simulator for training phlebotomy

Objective: The present study compared the effectiveness of a virtual reality (VR) simulator for training phlebotomy with that of a more traditional approach using simulated limbs. Background: Phlebotomy, or drawing blood, is one of the most common medical procedures; yet, there are no universal standards for training and assessing performance. The absence of any standards can lead to injuries and inaccurate test results if the procedure is improperly performed. Method: Twenty 3rd-year medical students were trained under one of the two methods and had their performance assessed with a 28-item checklist. Results: The results showed that performance improvements were limited to those who trained with the simulated limbs, and a detailed comparison of the two systems revealed several functional and physical differences that may explain these findings. Conclusion: Participants trained with simulated limbs performed better than those trained with a VR simulator; however, the metrics recorded by the VR system may address some aspects of performance that could eventually prove beneficial. Application: The present study highlights the potential for medical simulators to improve patient safety by enabling trainees to practice procedures on devices instead of patients. Applications of this research include training, performance assessment, and design of simulator systems.