Ernesto A. Bustamante

Effects of varying the threshold of alarm systems and workload on human performance

The purpose of this research was to investigate the effects of varying the threshold of alarm systems and workload on human response to alarm signals and performance on a complex task. A receiver operating characteristic (ROC) curve was selected to reflect the sensitivity of the alarm system. The threshold of the alarm system was manipulated by changing the value of beta along the ROC curve. A total of 84 students participated in experiment 1 and 48 students participated in experiment 2. Participants performed a compensatory-tracking, a resource management and a monitoring task. As expected, results showed that participants responded significantly faster to true alarm signals when they were using the system with the highest threshold under low-workload conditions. Results also indicated that changing the threshold of the alarm system had a significant effect on overall performance and this effect was greater under high-workload conditions. However, contrary to expectations, the highest level of performance was achieved by setting the threshold at a low level. Results from both experiments revealed that the advantage of faster alarm reaction time as a result of increasing the systems threshold was lost because of its increased probability of missed events.

Towards an Empirically Developed Scale for System Trust: Take Two

Research on human trust in automated systems has been frequently limited by the use of inappropriate and inaccurate scales of human-machine trust. An attempt is made here to validate a scale proposed by Jian, Bisantz, and Drury (2000) that measures trust and distrust in automated systems. Sixty participants completed a patient monitoring task with the aid of an imperfect signaling system. Confirmatory factor analyses were conducted using LISREL 8.0 to determine the best fitting measurement model. Chi-square difference tests indicated that the best fitting model was the 2-factor oblique model. These results suggest that when using the System Trust Scale, researchers should treat trust as a multi-dimensional construct comprising of two distinct, yet related, factors. Implications for future research and measurement development are discussed.

Effects of Varying the Threshold of Alarm Systems and Task Complexity on Human Performance and Perceived Workload

Using a dual-task paradigm, we examined how alarm system detection threshold and task complexity affected human performance and perceived workload. We hypothesized that using an alarm system would improve task performance and lower perceived workload, particularly when task complexity was high and at the medium threshold level. Twenty-one students from Old Dominion University participated in this study. Results showed that alarm use improved performance during low task complexity. For high task complexity, improvement was accomplished only when alarm system threshold was low or intermediate. Results also indicated that changing the alarm system threshold affected performance only under high task load conditions. Optimal performance was achieved by setting the threshold of the alarm system at its lowest level. The use of alarms reduced workload under both low and high task complexity levels, but only when the threshold was high.

Measurement Invariance of the Nasa TLX

Mental workload is one of the most important constructs of interests for Human Factors researchers. Adequately assessing the amount of mental workload that people experience while performing tasks under specific conditions is essential for the design of safe and efficient systems. Due to its ease of use, the NASA TLX has become the most widely used method of measuring mental workload. However, its psychometric properties are still questionable. The purpose of this study was to examine the extent of measurement invariance of the TLX and raise awareness in the Human Factors community. Two hundred participants reported the amount of mental workload they typically experience while driving in urban and rural areas and across the country. Results indicated that the TLX lacked scalar invariance, thereby biasing the estimation of mean scores and making the examination of mean differences misleading. These findings suggest that researchers should first examine the extent of measurement invariance of the TLX before they proceed to make inferences about mean differences in the amount of mental workload reported by participants under different conditions.

Enhancing Perceived Situation Awareness and Reducing Perceived Workload with a Likelihood Alarm Display

The researchers investigated the effects of a likelihood alarm display (LAD) on perceptions of workload and situation awareness (SA) during varying degrees of taskload. Twenty-four psychology students reacted to alarms while performing a complex primary task. The researchers examined participants performances during four experimental sessions. Two conditions of alarm display (binary and likelihood) and task load (low and high) were manipulated within groups. In addition, alarm display order (binary first or likelihood first) and task load order (low first or high first) were examined between groups. Results showed greater SA and reduced workload when participants used a likelihood alarm display and when they experienced low task load. Significant order effects also occurred. This study demonstrated the positive effects of a likelihood alarm display on perceptions of workload and SA, and suggests that LAD use may reduce workload and enhance SA.

Current Concepts and Trends in Human-Automation Interaction

The purpose of this panel was to provide a general overview and discussion of some of the most current and controversial concepts and trends in human-automation interaction. The panel was composed of eight researchers and practitioners. The panelists are well-known experts in the area and offered differing views on a variety of different human-automation topics. The range of concepts and trends discussed in this panel include: general taxonomies regarding stages and levels of automation and function allocation, individualized adaptive automation, automation-induced complacency, economic rationality and the use of automation, the potential utility of false alarms, the influence of different types of false alarms on trust and reliance, and a system-wide theory of trust in multiple automated aids.

The Effects of Relative System Reliability and Prioritization on Alarm Reaction Patterns

Alarm system operators often manage multiple alarm systems concurrently. Because such situations often accompany cascading events, it is important to know how operators sequence responses. We examined how the relative reliability and priority of two concurrent alarms affected alarm gauge reset patterns. We hypothesized that operators would respond first to an alarm with higher reliability or higher priority when the other variable was held constant. We also expected participants to respond to alarms with higher priority first when they occurred at the same time as a high-reliability alarm. Sixty-one Old Dominion University undergraduates performed a tracking task and responded to gauge fluctuations (alarms). A between subjects ANOVA revealed that participants responded to alarms with higher priority first when reliability was constant, and to higher reliability alarms first when priority was constant. Our results suggest that relative priority and reliability may be useful parameters to control in complex task situations.

Weather Deviation Decisions by Air Transport Crews during Simulated Flight

As the variety and capabilities of cockpit weather displays have increased, weather deviation decisions have become more complex. Important issues include conflicting or outdated weather information, and teamed reactivity of crews to weather displays. We investigated the influence of onboard and NEXRAD agreement, range to the simulated potential weather event, and the pilot flying on collective weather deviation decisions. Twelve pilot-copilot teams flew a simulated route while reacting to weather events presented in two graphical formats on a separate visual display. Results showed that pilots often chose to deviate from weather rather than confront it. When onboard and NEXRAD displays did not agree, flight crews reacted by trusting the onboard system more but using the NEXRAD system as a backup. These results suggest that future weather displays should exploit existing benefits of NEXRAD presentation for situation awareness while retaining the display structure and logic inherent in the onboard system.

Combating Cry Wolf: The Effects of Synthetic Verbal Alarm Urgency on Choice Reaction Performance:

Researchers have shown that increasing the perceived urgency of nonverbal auditory alarms can meliorate alarm response performance degradation due to the cry wolf effect. We conducted two experiments to examine the effects of verbal alarm urgency on the cry wolf effect. Experiment 1 revealed that participants responded significantly less often and correctly rejected more urgent alarms. Experiment 2 revealed that the results in Experiment 1 were contingent on the presence of additional task critical information. Response degradation due to the cry wolf effect was not moderated by alarm urgency in either study. However, the results suggest that performance may still benefit from increased alarm urgency provided additional task critical information is available. These results can be utilized to design verbal alarm systems that evoke more appropriate reactions to false alarms.