Ann M. Bisantz

Foundations for an Empirically Determined Scale of Trust in Automated Systems

One component in the successful use of automated systems is the extent to which people trust the automation to perform effectively. In order to understand the relationship between trust in computerized systems and the use of those systems, we need to be able to effectively measure trust. Although questionnaires regarding trust have been used in prior studies, these questionnaires were theoretically rather than empirically generated and did not distinguish between three potentially different types of trust: human-human trust, human-machine trust, and trust in general. A 3-phased experiment, comprising a word elicitation study, a questionnaire study, and a paired comparison study, was performed to better understand similarities and differences in the concepts of trust and distrust, and among the different types of trust. Results indicated that trust and distrust can be considered opposites, rather than different concepts. Components of trust, in terms of words related to trust, were similar across the three...

Making the abstraction hierarchy concrete

The abstraction hierarchy (AH) is a multileveled representation framework, consisting of physical and functional system models, which has been proposed as a useful framework for developing representations of complex work environments. Despite the fact that the AH is well known and widely cited in the cognitive engineering community, there are surprisingly few examples of its application. Accordingly, the intent of this paper is to provide a concrete example of how the AH can be applied as a knowledge representation framework. A formal instantiation of the AH as the basis for a computer program is presented in the context of a thermal-hydraulic process. This model of the system is complemented by a relatively simple reasoning mechanism which is independent of the information contained in the knowledge representation. This reasoning mechanism uses the AH model, along with qualitative user input about system states, to generate reasoning trajectories for different types of events and problems. Simulation outputs showing how the AH model can provide an effective basis for reasoning under different classes of situations, including challenging faults of various types, are presented. These detailed examples illustrate the various benefits of adopting the AH as a knowledge representation framework, namely: providing sufficient representations to allow reasoning about unanticipated fault and control situations, allowing the use of reasoning mechanisms that are independent of domain information, and having psychological relevance.

Integrating cognitive analyses in a large-scale system design process

This paper describes the integration of cognitive analysis into the early stages of design of a new, large-scale system--a next generation US Navy Surface combatant. Influencing complex system designs in ways cognizant of human-system integration principles requires work products that are timely and tightly coupled to other elements of the design process. Because analyses were conducted simultaneously with the design processes regarding ship functionality and staffing, it was necessary to select and adapt cognitive work analysis methods to fit the demands of a time pressured and information-limited design situation. Interviews were conducted and analyzed based on aspects of an abstraction hierarchy and control task models. An abstraction hierarchy, a series of cross-linked matrices, and a set of decision ladder models were developed to provide a principled mapping between system function decompositions produced by system engineering teams and cognitive tasks, information needs, automation requirements, and concepts for displays. Cross-referencing the matrices supported design traceability and facilitated the integration of cognitive analyses with functional analyses being performed by other design teams. Results fed into design recommendations with respect to level of automation, human roles and initial display prototypes for the ship combat command center. The case study illustrates the utility of cognitive work analysis models (specifically, abstraction hierarchies and decision-ladder models) in the design of large-scale, first-of-a-kind systems, and presents new design artifacts that link concepts used in cognitive analyses to those used in systems engineering for more effective integration within the systems engineering process.

Assessment of operator trust in and utilization of automated decision-aids under different framing conditions

Abstract Computerized aids may be used to support decision-making and control in a variety of complex, dynamic arenas. For instance, such systems have been introduced into industrial settings as the means to implement automated control or support decision-making activities such as fault detection and recovery. Of interest in these systems is the extent to which operators utilize and trust such systems, in terms of their ability to successfully control systems, or the information or decision support they provide, particularly under conditions of potential failure. A theoretical framework to describe potential factors affecting these issues, and an experiment to investigate the role of failure cause on trust and system utilization, are described. Results provide some support for factors in the theoretical framework, and also demonstrated the use of an empirically developed trust scale. Relevance to industry As manufacturing environments increasingly rely on computerized and automated systems for control and human operator support, it is necessary to understand the situational factors which could impact operators’ use of such systems. This paper describes a framework which could be used to investigate trust in industrial automation settings, as well as a rating scale which could be applied.

Utilizing graphical formats to convey uncertainty in a decision-making task

Understanding how to display effectively uncertain information has become increasingly important as decision aids can provide operators with situational estimates and their associated uncertainty. The paper describes two studies in which degraded or blended icons were used to convey uncertainty regarding the identity of a radar contact as hostile or friendly. A classification study first showed that participants could sort, order and rank icons from five sets intended to represent different levels of uncertainty. Three icon sets were selected for further study in an experiment in which participants had to identify the status of contacts as either hostile or friendly. Contacts and probabilistic estimates of their identities were depicted on a simulated radar screen in one of three ways: with degraded icons and probabilities, with non-degraded icons and probabilities and with degraded icons only. Results showed that participants using displays with only degraded icons performed better on some measures and as well on other measures, than the other tested conditions. These results are significant because they indicate both that people can understand uncertainty conveyed through such a manner and thus that the use of distorted or degraded images may be a viable alternative to convey situational uncertainty.

Displaying uncertainty : Investigating the effects of display format and specificity

We conducted four studies regarding the representation of probabilistic information. Experiments 1 through 3 compared performance on a simulated stock purchase task, in which information regarding stock profitability was probabilistic. Two variables were manipulated: display format for probabilistic information (blurred and colored icons, linguistic phrases, numeric expressions, and combinations) and specificity level (in which the number and size of discrete steps into which the probabilistic information was mapped differed). Results indicated few performance differences attributable to display format; however, performance did improve with greater specificity. Experiment 4, in which participants generated membership functions corresponding to three display formats, found a high degree of similarity in functions across formats and participants and a strong relationship between the shape of the membership function and the intended meaning of the representation. These results indicate that participants can successfully interpret nonnumeric representations of uncertainty and can use such representations in a manner similar to the way numeric expressions are used in a decision-making task. Actual or potential applications of this research include the use of graphical representations of uncertainty in systems such as command and control and situation displays.

Applications of cognitive work analysis

Applications of cognitive work analysis , Applications of cognitive work analysis , کتابخانه دیجیتالی دانشگاه علوم پزشکی و خدمات درمانی شهید بهشتی

Lessons From a Comparison of Work Domain Models: Representational Choices and Their Implications

As methods in cognitive work analysis become more widely applied, questions regarding the impact of modeling choices and similarities in modeling efforts across projects and domains are increasingly relevant. However, no explicit comparison of models of similar systems has been reported. This paper compares independently developed work domain analysis (WDA) models of two command and control environments. Similarities in model content and the types of nodes included provide evidence that WDA techniques can capture fundamental elements regarding purposes and constraints. These points of agreement provide a common starting point for developing work domain representations of military command and control systems. The comparison also revealed differences between the models. Although differences in content reflected differences in scope of coverage and level of detail, other differences corresponded to more fundamental choices in modeling approach. These included the treatment of sensors, level of integration in the model, and representation of particular abstract constraints. Examination of these more fundamental differences pointed to important degrees of freedom in how to represent a WDA and clarified the implications of these modeling choices for guiding design. Actual or potential applications of this research include aiding analysts in making work domain modeling choices as well as producing work domain models of command and control environments.

The effects of adverse condition warning system characteristics on driver performance: An investigation of alarm signal type and threshold level

This study addresses the issues concerning the design of adverse condition warning systems (ACWS). ACWS are designed to sense adverse road and weather conditions as well as system states that can negatively impact driving performance leading to skids or accidents, and alert drivers to these conditions. In this case, an ACWS was designed to sense when a car was likely to skid. A virtual-driving environment was used to test two levels of alarm sensitivity (low and high) and two types of auditory alarm signal (Binary ON/OFF and Graded) along with a no-alarm control group. Dependent measures reflected driver performance, response to the alarm signal and trust in the alerting system. Results indicated that participants had fewer skids in the low sensitivity and graded alarm signal condition compared to some other alerting system configurations. Participants in the graded alarm signal condition also had a greater degree of lateral control over the vehicle. Additionally, trust was found to be lower for the high vs. low sensitivity alarm condition, indicating a reduction in trust when the alerting system activated more often, perhaps because participants did not feel the system was accurately reflecting a dangerous condition. This simulator-based research emphasizes the fact that while ACWS may provide an advantage in terms of vehicle control, characteristics of both the alerting signal and system configuration should be considered.

Emergency Department Status Boards: A Case Study in Information Systems Transition:

Patient status boards play an important role in coordinating and communicating about patient care in hospital emergency departments (EDs). Status boards are transitioning from dry-erase whiteboards to electronic systems. Although electronic systems may preserve some surface features of the manual artifacts, important affordances of the manual technology are not always maintained. We compared information on manual and electronic status boards in an ED. Photographs of the manual board and screen shots of the electronic system were obtained before and after a hospital transitioned between systems. Displayed information as well as detailed content regarding patient chief complaints, clinical plans, and dispositions were coded and analyzed to understand the type and function of information present, as well as the use of features such color. Results indicated that although categories of information found were similar, the frequency with which some types of information appeared on the two system displays was substantially different. In particular, information used to coordinate aspects of patient treatment was more frequently found in the manual system. Results suggest that in the design of new information technologies, simply matching the format or information fields available on an existing system may not be sufficient to sustain current work practices or to prevent unanticipated shifts in use.