Susan G. Hutchins

Radio Interoperability: There is More to it Than Hardware

Following 9/11, many public safety officials reported communications issues between first responders, echoing concerns predating the September 11, 2001 terrorist attacks. With increased availability of grant funds for homeland security, radio interoperability has become a top priority. The rush to address emergency radio interoperability has caused agencies to select equipment without consideration given to the human aspects of communicating during a crisis and proper communication device ergonomics. This paper presents the results of a field study, providing insight into typical communications inefficiencies and opportunities to maximize limited resources via improved procedures and product design.

Use of Critical Analysis Method to Conduct a Cognitive Task Analysis of Intelligence Analysts

Intelligence analysts (IAs) engage in information seeking, evaluation, prediction, and reporting behavior in an information-intensive work environment. A Cognitive Task Analysis (CTA) was conducted to capture data that will provide input to support development of a model of the IAs processes, biases, and analytic strategies. A hybrid method was used to conduct the CTA, including a modified version of the critical decision method — the critical analysis method. The essential distinction was that participants were asked to describe an example of a strategic analysis problem. Procedures used to conduct the critical analysis method are described in this paper. Several factors contribute to making the IAs task challenging: (i) time pressure to produce reports in a shorter timeframe, (ii) a high cognitive workload, and (iii) difficult human judgments that are required regarding uncertain validity and reliability of the data. Human judgments are involved in considering the plausibility of information, deciding what information to trust, and determining how much weight to place on specific pieces of data. Intelligence analysis involves a complex process of assessing the reliability of information from a wide variety of sources and combining seemingly unrelated events. This problem is challenging because it involves aspects of data mining, data correlation and human judgment.

Analysis of Human Factors Case Studies of Complex Military Systems: Here's How We Can do Better

People in nearly every occupational setting can provide examples of poor system design. The focus for this paper is on an analysis of design problems found in complex military command and control systems and the ways in which these types of problems can be avoided in future system design. The source of data for this analysis was a group of case studies of sixteen U.S. military systems written by officer-students at the Naval Postgraduate School, Monterey, CA. Systems analyzed include aircraft systems, communications systems, the M-16 rifle, a missile defense system, a message processing system, weapon systems, and decision support systems. Documented problems with system use were categorized according to the following measures of effectiveness: Performance, Safety, Usability, Reliability, Maintainability, Time and Cost to Train, and Workload. The number of problems encountered per system ranged from one to nine; the mean number of reported problems per system was 4.9. IEEE 1220-1998 includes a revised systems engineering approach with an increased emphasis on engineering the system for the human. Adhering to a user-centered design approach would have a positive impact on system design by significantly reducing the types of system problems described in this paper.

Analysis of human factors case studies of complex military systems : Surely we can do better

People in nearly ever occupational setting can provide examples of poor system design. The focus for this paper is on an analysis of design problems found in complex military command and control systems and the ways in which these types of problems can be avoided in future system design. The source of data for this analysis was a group of case studies of sixteen U.S. military systems written by officer-students at the Naval Postgraduate School, Monterey, CA. Systems analyzed span the four military services and include aircraft systems, communications systems, the M-16 rifle, a missile defense system, a message processing system, weapon systems, and decision support systems. Documented problems with system use were categorized according to the following measures of effectiveness: Performance, Safety, Usability, Reliability, Maintainability, Time and Cost to Train, and Workload. The number of problems encountered per system ranged from one to nine; the mean number of reported problems per system was 4.9 IEEE 1220-1998 includes a revised systems engineering approach with an increased emphasis on engineering the system for the human. Adhering to a user-centered design approach should have a positive impact on system design by significantly reducing the types of system problems described in this paper.

Patterns of Team Collaboration Employed to Solve Unique Problems

“Macrocognition” is an emerging area of knowledge engineering that focuses on understanding how cognition emerges in natural environments. One goal for studying macrocognition is to understand the complexity entailed in inter- and intra-individual cognition. In this paper we describe our analysis of several complex team collaboration tasks: firefighters from the Fire Department of New York on Sept. 11, 2001, air warfare teams on an Aegis ship, and the team collaboration entailed in conducting Maritime Interdiction Operations. Team communications that transpired during three complex problem solving situations were analyzed to understand how teams collaborate to create new knowledge and decide on a course of action during complex, one-of-a-kind problems. These processes include (1) individual knowledge building, (2) developing knowledge interoperability, (3) team shared understanding, and (4) team consensus. The way the teams cognitive behavior maps to the model is discussed along with differences in patterns of collaboration for different decision-making domains.

Measures of effectiveness under different command and control organizational architectures

Todays military faces enormous pressure to downsize and restructure. In addition, operations-other-than-war, such as humanitarian assistance and peace-keeping, differ significantly from traditional combat missions. The unique characteristics of peace-keeping missions involve inherently more complex command arrangements. Moreover, the increased political sensitivity associated with these missions further exacerbates the stress on command and control (C2) arrangements. Thus, Joint and Coalition doctrine is more complex and reliant on the synchronized employment of combined arms. The process of organizing for Joint and Coalition operations must be driven by the disparate missions and the requisite tasks involved. Depending on the specific mission, the diverse range of future anticipated operations can require a wide variety of service capabilities under a Joint Force Coalition. It follows that the C2 organization should be flexible in order to allow commanders to meet the many and diverse operations that are expected to continue to undertaken. This paper reports on the use of measurement instruments developed for the initial experiment conducted under the Adaptive Architectures for Command and Control (A2C2) program. The goal of the A2C2 research effort is to provide insights based on exploring innovative thinking and empirical research on organizational design that can assist in positioning the Joint community to face the diverse challenges and dynamic changes that are projected for the future.

Understanding Cognition in Team Collaboration through use of Communications Analysis

Macrocognition has been conceptualized by researchers in a number of ways depending on the perspective of the different disciplines. For this research the focus is on cognition in collaboration contexts, where problem solving teams collaborate on short-term situations which require rapid action to be taken against specific missions. We employed an empirical process for evaluating a model of team collaboration by analyzing and coding transcripts or chat logs that transpired during several real-world problem-solving events. Traditional collaboration systems are domain independent, thus they do not always support the difficult cognitive aspects of the task. The goal for the research reported here is to understand the role of cognition in teams who are collaborating to solve challenging, unique, information-rich problems and to apply this understanding to make recommendations for collaboration support.

USER-CENTERED INTERFACE DESIGN OF TACTICAL AIRCRAFT DISPLAYS

Tactical aviators must perform under highly complex, dynamic, multitask decisionmaking situations. At any point in time a number of different tasks may demand attention, and each of these tasks is typically knowledge intensive and procedurally complex. Contact identification alone often involves time-compressed, ambiguous decision problems. The memory and cognitive processing requirements of these tasks often exceed the humans capacity to perform the requisite subtasks. A user-centered design approach was taken with deference to human performance capacities and limitations. Design features include the use of graphics to support intuitive processes and reduce cognitive processing requirements and structuring and presenting information in a format that parallels the decisionmakers natural cognitive strategies. These decision support display features aredirected at improving information sampling, reducing processing complexity, making information conform to human memory and cognitive strategy characteristics, avoiding cognitive overload, and enhancing the speed, coherence and efficiency with which decisionmakers are able to shift attention and thought between the various tasks that arise. Topics include a discussion of decision support and human-system interaction principles and examples of modules designed to enhance decisionmaking. INTRODUCTION Tactical air warfare spans a broad class of actual scenarios, including both offensive and defensive operations for land and sea-based assets. In spite of the differences in mission, tactics and weaponry inherent in these different scenarios, however, the need for automated decision support technology to assist the decisionmaker when time is short, when contacts are numerous, and when the situation is both fluid and ambiguous is growing for all such operations. Furthermore, the principles underlying this decision support technology are much more unified than the disparate missions would suggest. All involve high cognitive workloads, short decision times, and potentially catastrophic consequences for bad decisions. In recognition of the complex and difficult decisions required in these types of situations the Tactical Decision Making Under Stress (TADMUS) program was initiated to conduct research in the areas of human factors and training technology. The TADMUS program is being conducted to apply recent developments in decision theory and humansystem interaction technology to the design of a decision support system for enhancing tactical decisionmaking under the highly complex conditions involved in antiair warfare scenarios in close to land environments. The goal is to present decision support information in a format that minimizes any mismatches between the cognitive characteristics of the human decisionmaker and the design and response characteristics of the decision support system. This paper presents these underlying decision support principles and discusses their use in the design of decision support systems (DSS) for enhanced tactical air situation awareness in airborne, land and sea-based systems. Much of the technical data presented below was developed in conjunction with a DSS specifically designed for implementation aboard Aegis cruisers to assist decisionmakers in the combat information center (CIC) in responding to antiair scenarios. Although not designed for airborne missions, such a DSS is directly applicable to the type of missions flown by airborne warning and control system (AWACS) aircraft, (as well as other aircraft) and indeed a fielded system might have prevented the tragic mistake in Iraq when a U.S. AWACS aircraft vectored two fighters to shoot down hostile helicopters that eventually proved to be U.S. Army helicopters carrying U.S. and foreign dignitaries. In jumping from decisionmaking in the team environment characteristic of an Aegis CIC or an AWACS aircraft to the lonelier confines of a fighter or bomber cockpit, the main difference in terms of the principles underlying a DSS is that there is no longer ambiguity and error created in the. communication process between the team members and the decisionmaker. Other factors, such as multiple and ambiguous contacts, high task loads on the individual, short decision time frames, ambiguous and erroneous information from off-board sources, and time-critical responses, are similar across all aspects of tactical air operations. A discussion of the common underlying principles in decision support will comprise the remainder of this section.