Douglas S. Lange

Design Patterns for Human-Cognitive Agent Teaming

The aim of this article is to provide a common, easy to use nomenclature to describe highly automated human-machine systems in the realm of vehicle guidance and foster the identification of established design patterns for human-autonomy teaming. With this effort, we intend to facilitate the discussion and exchange of approaches to the integration of humans with cognitive agents amongst researchers and system designers. By use of this nomenclature, we identify most important top-level design patterns, such as delegation and associate systems, as well as hybrid structures of humans working with cognitive agents.

A task analysis toward characterizing cyber-cognitive situation awareness (CCSA) in cyber defense analysts

Cyberspace is an increasingly crucial part of everyday living. We have long recognized that defending this space is complex, requiring information integration, and decisions of man and machine to coalesce in a dynamic environment full of shifting priorities. These properties suggest that, as in other domains with similar characteristics, situation awareness (SA) of a human cyber defender is paramount to the quality of decision outcomes in cyber defense. The majority of existing research in cyber situation awareness, centers on information systems and computers, which piece together disparate data. Fused data from multiple sources, for example, is necessary for cyberspace visualization efforts. The judgment for successful cyber SA from this perspective is different from one that is human-centered. In comparison, we rarely assess human cognitive awareness in cyberspace. In part, this reflects a need, based on prior theory, to first define critical elements of information that the human must perceive, work to elucidate how humans combine these elements to comprehend the state of the network, and how together, this awareness helps analysts predict the future state of the network. In other words, although data fusion can provide value by reducing the cognitive load created to piece together disparate sources of information, human awareness of the network (cyber-cognitive situation awareness - CCSA) is perhaps the ultimate intermediary for defense performance. Toward such an understanding, we discuss the results of a cognitive task analysis (CTA) which sought to determine the goals and abstracted elements of awareness that cyber analysts seek in network defense. We present the foundation for a series of planned experiments that establishes CCSA measurement, and baselines the efforts of cyber defenders. Once assessed, we can then begin to consider the help offered by fusion systems, automation of defensive capabilities, and cyber visualizations in a methodologically rigorous manner that has been lacking.

Human-Computer Collaboration in Adaptive Supervisory Control and Function Allocation of Autonomous System Teams

The foundation for a collaborative, man-machine system for adaptive performance of tasks in a multiple, heterogeneous unmanned system teaming environment is discussed. An autonomics system is proposed to monitor missions and overall system attributes, including those of the operator, autonomy, states of the world, and the mission. These variables are compared within a model of the global system, and strategies that re-allocate tasks can be executed based on a mission-health perspective (such as relieving an overloaded user by taking over incoming tasks). Operators still have control over the allocation via a task manager, which also provides a function allocation interface, and accomplishes an initial attempt at transparency. We plan to learn about configurations of function allocation from human-in-the-loop experiments, using machine learning and operator feedback. Integrating autonomics, machine learning, and operator feedback is expected to improve collaboration, transparency, and human-machine performance.

Command and control of teams of autonomous systems

The command and control of teams of autonomous vehicles provides a strong model of the control of cyber-physical systems in general. Using the definition of command and control for military systems, we can recognize the requirements for the operational control of many systems and see some of the problems that must be resolved. Among these problems are the need to distinguish between aberrant behaviors and optimal but quirky behaviors so that the human commander can determine if the behaviors conform to standards and align with mission goals. Similarly the commander must able to recognize when goals will not be met in order to reapportion assets available to the system. Robustness in the face of a highly variable environment can be met through machine learning, but must be done in a way that the tactics employed are recognizable as correct. Finally, because cyber-physical systems will involve decisions that must be made at great speed, we consider the use of the Rainbow framework for autonomics to provide rapid but robust command and control at pace.

A Design Pattern for Working Agreements in Human-Autonomy Teaming

Humans and machines are increasingly reliant upon each other in complex environments and military operations. The near future suggests human understanding of machine counterparts is a required, paradigmatic element. Knowing how to engineer and design for these environments is challenging. The complexity between levels of automation, human information processing, and function allocation authority issues in an adaptive system make it unlikely to find a “one-size-fits-all” approach. There may still be general strategies for engineering in these cases; for example, collaborating and coordinating are familiar requirements of all human team activities, and extend to human-automation teaming. Here, we outline what we believe is one so-called “design pattern” for working agreements. We use the loose structure of prior software design patterns to organize our thoughts on why working agreements are necessary, where and how they are applicable, what instantiating them requires, and how to measure their effectiveness. By choosing the design pattern structure, we end up carefully describing what might work best and what the limits are toward improving human-machine teaming.

Human-Autonomy Teaming Patterns in the Command and Control of Teams of Autonomous Systems

Design patterns have been found useful in several domains. This paper helps motivate their use in the field of human-autonomy teaming and provides three example patterns that could be contributed to the language of patterns available to system developers. In our examples we focus on the motivations and consequences in terms of human and team performance when describing the features of the individual patterns. These replace forces more commonly used in software engineering or other fields. Practitioners and researchers alike will benefit from a vetted vocabulary of established patterns of the form presented.

Tasking Teams: Supervisory Control and Task Management of Autonomous Unmanned Systems

How does one collaborate with and supervise a team? Here, we discuss a novel interface for managing tasks, developed as part of a multi-heterogeneous unmanned systems testbed, that aids cognitive operations and teaming. Existing models of team effectiveness among humans can frame cooperative teaming of computer agents and human supervisors. We use the three main characteristics of the input – process – output model to frame discussions of the task manager interface as a potential teaming facilitator, finding it should facilitate effectiveness on several elements. We conclude with the expectation of examination and support from future experiments.

Tracking Provenance in Decision Making Between the Human and Autonomy

Provenance has been used as a measure of accountability, trust, and validity. Within the context of Command and Control (C2), provenance can be utilized to track the decision making processes that change data and dependencies. C2 of several unmanned autonomous vehicles provide a complex and ever-changing battlespace, with many actors and decision makers. The goal of this paper is to track and explain the autonomous decisions made by an intelligent C2 station based on its interactions with various systems using provenance. With provenance providing explanations and reasoning behind the actions of autonomy, a form of system accountability and transparency is achieved between human and machine.

A Heterarchical Urgency-Based Design Pattern for Human Automation Interaction

We document a Human-Autonomy Teaming design pattern to provide a means for an task management assistant to mitigate errors that may occur due to changes in urgency levels of tasks. Urgency can increase or decrease due to changes in the task environment, or through failure to begin execution of a task at the correct time. We discuss the structure and key aspects of the pattern and provide a sample implementation. We also discuss the key aspects of the human partner’s performance that must be measured and considered in implementing such a pattern. Finally, we discuss known issues and other related patterns.

Amplifying human ability through autonomics and machine learning in IMPACT

Amplifying human ability for controlling complex environments featuring autonomous units can be aided by learned models of human and system performance. In developing a command and control system that allows a small number of people to control a large number of autonomous teams, we employ an autonomics framework to manage the networks that represent mission plans and the networks that are composed of human controllers and their autonomous assistants. Machine learning allows us to build models of human and system performance useful for monitoring plans and managing human attention and task loads. Machine learning also aids in the development of tactics that human supervisors can successfully monitor through the command and control system.