William C. Elm

Getting Lost: A Case Study in Interface Design

This paper describes an application of principles of spatial data management to correct for an information handling problem, the “getting lost” phenomena in a large multiple display network. The application is the computerization (e.g., electronic book) of a book of procedures for use by nuclear power plant operators during accident conditions.

FINDING DECISION SUPPORT REQUIREMENTS FOR EFFECTIVE INTELLIGENCE ANALYSIS TOOLS

Within ARDAs GI2Vis program, we developed a unique framework for the definition of decision support requirements for intelligence analysis tools. This framework, based on a first-of-a-kind integration of a model of inferential analysis and principles for designing effective human-computer teams from Cognitive Systems Engineering, has defined the essential support functions to be provided to the intelligence analyst(s). This model has proven to be extremely useful in assessing the support provided by a large set of visualization tools. This assessment has identified clusters of support functions that are addressed by many tools as well as key missing support functions. In this way, the Support Function Model has been used to identify gaps in the support function coverage of existing tools. This can serve as a valuable focusing mechanism for future design and development efforts. In addition, we believe this would be a useful mechanism to enhance cross-discussions among research teams involved in Cognitive Task Analysis efforts within the Intelligence Community. Having others integrate their analytic results with this framework would provide the mechanism for expansion of this model to become a more robust tool and have an even greater impact on the Intelligence Community.

Cognitive System Engineering - Based Design: Alchemy or Engineering

Cognitive Systems Engineering (CSE) techniques are widely used for the description and analysis of the sources of cognitive complexity and explicating the basis of expertise within a work domain. However, the results of the CSE techniques often focus on work analysis and are only weakly coupled to the design of decision support systems that are built based on those analyses. In fact, some within the CSE community have suggested that such a design epiphany occurs as if by magic. If CSE is to be treated as an engineering discipline, it cannot rely on magic to create systems. The approach described in this paper assumes that an explicit relationship between system design and supported cognitive work is fundamental to the designs effectiveness. The goal is a pragmatic, effective engineering process that explicitly designs systems according to relationships between cognitive work requirements and decision support concepts.

Scenario Development for Decision Support System Evaluation

This paper introduces a methodology for developing scenarios representative of the cognitive and collaborative challenges inherent in a domain of practice for evaluating Decision Support Systems (DSS). Explicit links are made between particular aspects of the DSS and specific cognitive and collaborative demands they are intended to support. The effectiveness of the DSS in supporting performance can then be systematically probed by creating scenarios that are informed by an understanding of individual and team cognitive processing factors, fundamental relationships within the domain, and known complicating factors that can arise in the domain to challenge cognitive and collaborative performance. This paper introduces a set of explicit artifacts to systematically create such scenarios to provide feedback on the viability of the DSS design concepts (e.g., are the hypothesized positive impacts of the DSS realized?), as well as feedback on additional unanticipated requirements for support.

A Framework for Integrating Cognitive Task Analysis into the System Development Process

This paper describes a process that orchestrates different types of specific CTA techniques to provide design relevant CTA results and integrates CTA results into the software development process. Two fundamental premises underlie the approach. First, CTA is more than the application of any single CTA technique. Instead, developing a meaningful understanding of a field of practice relies on multiple converging techniques in a bootstrapping process. The important issue from a CTA perspective is to evolve a model of the interconnections between the demands of the domain, the strategies and knowledge of practitioners, the cooperative interactions across human and machine agents, and how artifacts shape these strategies and coordinative activities across a series of different specific techniques. Second, since CTA is a means to support the design of computer-based artifacts that enhance human and team performance, CTA must be integrated into the software and system development process. Thus, the vision of CTA as an initial, self-contained technique that is handed-off to system designers is reconceived as an incremental process of uncovering the cognitive demands imposed on the operator(s) by the complexities and constraints of the domain.

Analysis with a Purpose: Narrowing the Gap with a Pragmatic Approach

There has been a growing interest in using Cognitive Systems Engineering (CSE) techniques to understand work domains and the cognitive demands they imposes on practitioners in order to provide a foundation for the design of decision-aids. While CSE techniques, like Cognitive Work Analysis (CWA), have been proven successful in illuminating the sources of cognitive complexity and explicating the basis of expertise, there is often still a gap between the results of the CWA and the resulting design and development of the decision support system. One way to narrow the gap is to develop an integrated set of artifacts that provide explicit links between (1) the functional goals the domain, to (2) the cognitive demands that require support, through (3) the mapping of decisions to the display space. In this paper a brief discussion of a recent example where this approach was taken is presented.

Behind the Curtain: The Cognitive Tasks behind the Visualizations

This panel includes participants from academia and industry who have each made significant contributions to the design of effective visualizations to support decision making in a variety of domains. Panel members will offer an example of an innovative decision support visualization concept and discuss the underlying cognitive demands it is meant to support as well as any artifacts they used in its development. Both the nature of the visualization itself, and the linkage to the processes ‘behind the curtain’ will be presented. Panelists will discuss the various techniques used and the pros and cons of each.

Power Tool for Countering Cyberwar: Visualizations for Information Assurance and Computer Network Defense

There has been a growing need for military decision-makers to maintain the integrity of the information contained within their computer network. Tools to support Information Assurance and Computer Network Defense (IA-CND) are needed to defend their information infrastructure and conduct Computer Network Operations with a new level of insight and understanding. This paper describes one effort to develop visualizations to aid these decision-makers in the highly abstract, complex and dynamic mission of IA-CND. This paper describes the development of a IA-CND Communications Display. Using a Cognitive Systems Engineering methodology, this project transitioned from a broad description of a work domain, to the development of decision aiding concepts for a particular portion of that domain. This methodology also provided a means to develop breakthrough support for a decision difficult domain.

Providing On-Line Advice for a Dynamic Control Task: A Case Study in Intelligent Support System Design

This paper describes an Intelligent Manual Feedwater Control Station (IMFCS) that provides on—line expert guidance for a process control task: control of feedwater during power plant startup and operation. The IFMCS provides control action advice derived from analysis of the performance of expert operators. It represents a novel approach to advisory systems in that (a) the IMFCS does not attempt to replicate in detail the surface activities of domain experts, but rather abstracts the cognitive competencies required for expert performance and provides a domain representation that promotes those competencies; (b) the advice takes an analogue graphic display form; (c) a common frame of reference is established between machine advisor and human practitioner that integrates advice and “explanation” avoiding problems associated with opaque advice.

Bridging the Gap between Cognitive Systems Engineering Analysis, Design and Practice

Cognitive Systems Engineering principles and methods guide analysis and understanding of complex work domains. Taking the CSE analytic findings to the next step, design, is often not clearly specified in process form. Some reasons for a weakly specified and documented CSE analysis-to-design translation process are: (1) variations in work domain, making generalizations difficult; (2) the value of the analyses in relation to the scope of the study – analyses from better funded studies create more value propositions for a client; and (3) practical resource considerations. The panelists share their insights on tackling challenges in translating results from CSE analysis into impactful and practicable designs, and offer solutions and strategies for making effective CSE recommendations for practice.