Jamie C. Gorman

Measuring team situation awareness in decentralized command and control environments

Decentralized command and control settings like those found in the military are rife with complexity and change. These settings typically involve dozens, if not hundreds to thousands, of heterogeneous players coordinating in a distributed fashion in a dynamically networked battlefield laden with sensor data, intelligence reports, communications, and plans emanating from many different perspectives. Consider the concept of team situation awareness in this setting. What does it mean for a team to be aware of a situation or, more importantly, of a critical change in a situation? Is it sufficient or necessary for all individuals on the team to be independently aware? Or is there some more holistic awareness that emerges as team members interact? We re-examine the concept of team situation awareness in decentralized systems beyond an individual-oriented knowledge-based construct by considering it as a team interaction-based phenomenon. A theoretical framework for a process-based measure called ‘coordinated awareness of situations by teams’ is outlined.

Interactive team cognition.

Cognition in work teams has been predominantly understood and explained in terms of shared cognition with a focus on the similarity of static knowledge structures across individual team members. Inspired by the current zeitgeist in cognitive science, as well as by empirical data and pragmatic concerns, we offer an alternative theory of team cognition. Interactive Team Cognition (ITC) theory posits that (1) team cognition is an activity, not a property or a product; (2) team cognition should be measured and studied at the team level; and (3) team cognition is inextricably tied to context. There are implications of ITC for theory building, modeling, measurement, and applications that make teams more effective performers.

Team cognition in experienced command-and-control teams.

Team cognition in experienced command-and-control teams is examined in an UAV (Uninhabited Aerial Vehicle) simulation. Five 3-person teams with experience working together in a command-and-control setting were compared to 10 inexperienced teams. Each team participated in five 40-min missions of a simulation in which interdependent team members control a UAV to take reconnaissance photos. Experienced teams exceeded performance of inexperienced teams, suggesting transfer of previous command-and-control experience. Compared to inexperienced teams, experienced teams had fewer errors on process-related training knowledge, superior team process ratings, and communications containing fewer coordination-related utterances. These findings support the view that team cognition emerges through the interactions of team members, that interactions distinguish high-performing teams from average teams, and that these interactions transfer across different tasks.

Training Adaptive Teams

Objective: We report an experiment in which three training approaches are compared with the goal of training adaptive teams. Background: Cross-training is an established method in which team members are trained with the goal of building shared knowledge. Perturbation training is a new method in which team interactions are constrained to provide new coordination experiences during task acquisition. These two approaches, and a more traditional procedural approach, are compared. Method: Assigned to three training conditions were 26 teams. Teams flew nine simulated uninhabited air vehicle missions; three were critical tests of the team’s ability to adapt to novel situations. Team performance, response time to novel events, and shared knowledge were measured. Results: Perturbation-trained teams significantly outperformed teams in the other conditions in two out of three critical test missions. Cross-training resulted in significant increases in shared teamwork knowledge and highest mean performance in one critical test. Procedural training led to the least adaptive teams. Conclusion: Perturbation training allows teams to match coordination variability during training to demands for coordination variability during posttraining performance. Although cross-training has adaptive benefits, it is suggested that process-oriented approaches, such as perturbation training, can lead to more adaptive teams. Application: Perturbation training is amenable to simulation-based training, where perturbations provide interaction experiences that teams can transfer to novel, real-world situations.

Interaction-Based Measures of Cognitive Systems

Integrating cognitive engineering into the systems engineering process requires measurement methodologies that capitalize on variability in cognition and behavior distributed across people and their environment. It is important for cognitive system integration that measures be not only reliable and valid but also unobtrusive and capable of providing predictive and diagnostic information in real time. To achieve these objectives we have developed measures of systems (in our case, small teams of humans) that are based heavily on the automated sequential analysis of communication data. These measures have been mapped to system performance, system change, system process, coordination, and situation awareness and thus are of potential value for cognitive system integration activities.

Some Promising Results of Communication-Based Automatic Measures of Team Cognition

Some have argued that the most appropriate measure of team cognition is a holistic measure directed at the entire team. In particular, communication data are useful for measuring team cognition because of the holistic nature of the data, and because of the connection between communication and declarative cognition. In order to circumvent the logistic difficulties of communication data, the present paper proposes several relatively automatic methods of analysis. Four data types are identified, with low-level physical data vs. content data being one dimension, and sequential vs. static data being the other. Methods addressing all four of these data types are proposed, with the exception of static physical data. Latent Semantic Analysis is an automatic method used to assess content, either statically or sequentially. PRONET is useful to address either physical or content-based sequential data, and we propose CHUMS to address sequential physical data. The usefulness of each method to predict team performance data is assessed.

Measuring patterns in team interaction sequences using a discrete recurrence approach.

Objective: Recurrence-based measures of communication determinism and pattern information are described and validated using previously collected team interaction data. Background: Team coordination dynamics has revealed that “mixing” team membership can lead to flexible interaction processes, but keeping a team “intact” can lead to rigid interaction processes. We hypothesized that communication of intact teams would have greater determinism and higher pattern information compared to that of mixed teams. Method: Determinism and pattern information were measured from three-person Uninhabited Air Vehicle team communication sequences over a series of 40-minute missions. Because team members communicated using push-to-talk buttons, communication sequences were automatically generated during each mission. Results: The Composition × Mission determinism effect was significant. Intact teams’ determinism increased over missions, whereas mixed teams’ determinism did not change. Intact teams had significantly higher maximum pattern information than mixed teams. Conclusion: Results from these new communication analysis methods converge with content-based methods and support our hypotheses. Application: Because they are not content based, and because they are automatic and fast, these new methods may be amenable to real-time communication pattern analysis.

Changes in team cognition after a retention interval: the benefits of mixing it up.

This paper examines the retention of team cognition with changes in team membership. Hypotheses are developed from shared cognition and interactive team cognition theories. We report a study of the effects of Short (3-6 weeks) versus Long (10-13 weeks) retention intervals and change (Mixed) versus no change (Intact) in team membership during the interval on shared knowledge, team process, and team performance. The study context was a three-person Unmanned Aerial Vehicle (UAV) simulator. The long retention interval resulted in significantly lower team process scores and, except for the Short-Intact condition, all teams suffered a drop in performance after the break. However, those teams recovered prebreak levels of performance after one UAV mission. The counterintuitive result was that team mixing resulted in significant knowledge and process gains. An exploratory communication analysis indicated that Mixed team communication is longer in duration than Intact team communication, and Long-interval teams communicated more frequently than Short-interval teams. Unlike the Long-interval communication frequency effect, the Mixed team communication duration effect lasted throughout the experiment, suggesting greater interaction experience for Mixed teams. An exploratory mediation analysis indicated that the shared cognition Input-Process-Output framework was a good fit for the Intact team data, but not for the Mixed team data. We conclude that there are team-learning benefits of team mixing and that the interactive team cognition theory accounts better for those benefits than shared cognition theory.

Team Coordination and Dynamics Two Central Issues

When people coordinate as a team, they accomplish more than they would working alone. These team-coordination effects give new meaning to Aristotle’s famous phrase, “the whole is greater than the sum of its parts.” In this article, I consider two central issues confronting team-coordination research: Do the causes of team coordination reside within individual minds or between them, and at what levels of analysis (e.g., physiological, cognitive) do team-coordination effects occur? These issues are viewed in light of specific lines of coordination research, and some features of a general theory of team coordination are offered.

Coordinated Awareness of Situation by Teams (CAST): Measuring Team Situation Awareness of a Communication Glitch

A coordination-based measure of team situation awareness is presented and contrasted with knowledge-based measurement. The measure is applied to team awareness of a communication channel failure (glitch) during a simulated unmanned air vehicle reconnaissance experiment. Experimental results are reported, including the findings that not all team members should be identically aware of the glitch and that appropriate levels of coordination are an important precursor of team situation awareness. The results are discussed in terms of the application of coordination metrics to support the understanding of team situation awareness. The use of team coordination as a low-dimension variable of team functionality is scalable over a variety of team sizes and expertise distributions.