Axel Schulte

System-Ergonomic Design of Cognitive Automation

Motivation and Purpose of This Book.- Introductory Survey on Operational Guidance and Control Systems.- Basics about Work and Human Cognition.- Dual-Mode Cognitive Automation in Work Systems.- Examples of Realisations of Cognitive Automation in Work Systems.- Implementation Examples of Crucial Functional Components of Cognitive Automation.- Operationalisation of Cognitive Automation in Work Systems.

Managing Cockpit Crew Excess Task Load in Military Manned-Unmanned Teaming Missions by Dual-Mode Cognitive Automation Approaches

This work addresses approaches to enable the cockpit crew of a helicopter to manage multiple Uninhabited Aerial Vehicles (UAVs) while at the same time performing a military flight mission with the own air vehicle. Aiming for more flexibility and safety in combat, such Manned-Unmanned Teaming (MUM-T) missions require a high level of interoperability between manned and unmanned assets. The resulting additional responsibilities pose extreme work demands to the cockpit crew. The key elements for achieving the workload reduction necessary to facilitate such missions are threefold: first, high-level human-UAV interaction based on tasks (supervisory control), second, the delegation of helicopter commander tasks to the pilot flying (modified crew coordination concept), and third, the introduction of knowledge-based assistant systems, one for each crewmember to support their individual task performance (cooperative control). We implemented the mentioned functionalities along the design guidelines of the so-called DualMode Cognitive Automation approach, in which the functions that the human operator interacts with in supervisory control (mode one) and cooperative control (mode two) are realized as artificial cognitive systems. These systems make use of cognitive task understanding, human-machine mixed-initiative interaction, human operator observation, and human mental resource prediction approaches. In addition to the concepts and methods behind the functionalities mentioned, this article describes comprehensive evaluation experiments conducted in our research helicopter mission simulator and discusses the corresponding results.

Design Patterns for Human-Cognitive Agent Teaming

The aim of this article is to provide a common, easy to use nomenclature to describe highly automated human-machine systems in the realm of vehicle guidance and foster the identification of established design patterns for human-autonomy teaming. With this effort, we intend to facilitate the discussion and exchange of approaches to the integration of humans with cognitive agents amongst researchers and system designers. By use of this nomenclature, we identify most important top-level design patterns, such as delegation and associate systems, as well as hybrid structures of humans working with cognitive agents.

Human factors in the guidance of uninhabited vehicles: oxymoron or tautology?: The potential of cognitive and co-operative automation

Today’s automation is typically tied into work processes as tools actively supporting the human operator in fulfilling certain well-defined sub-tasks. The human operator is in the role of the high-end decision component determining and supervising the work process. With emergent technology highly automated work systems can be beneficial on the one hand, but automation may as well cause problems on its own. A new way of introducing automation into work systems shall be advocated by this article overcoming the classical pitfalls of automation and simultaneously taking the benefit as wanted. This shall be achieved by so-called cognitive automation, i.e. providing human-like problem-solving, decision-making and knowledge processing capabilities to machines in order to obtain goal-directed behaviour and effective operator assistance. A key feature of cognitive automation is the ability to create its own comprehensive representation of the current situation and to provide reasonable action. By additionally providing full knowledge of the prime work objectives to the automation it will be enabled to co-operate with the human operator in supervision and decision tasks, then being intelligent machine assistants for the human operator in his work place. Such assistant systems understand the work objective and will be heading for the achievement of the overall desired work result. They will understand the situation (e.g. opportunities, conflicts) and actions of team members—whether humans or assistant systems—and will pursue goals for co-operation and co-ordination (e.g. task coverage, avoidance of redundancy or team member overcharge). On the other hand, cognitive automation can be emerged towards being highly automated intelligent agents in charge of certain supportive tasks to be performed in a semi-autonomous mode. These cognitive semi-autonomous systems and the cognitive assistants shall be denoted as the two faces of dual-mode cognitive automation (Onken and Schulte, in preparation).

Efficient Graph Matching with Application to Cognitive Automation

Cognitive automation has proven to be an applicable approach to handle increasing complexity in automation. Although fielded prototypes have already been demonstrated, the real time performance of the underlying software framework COSA is currently a limiting factor with respect to a further increase of the application complexity. In this paper we describe a cognitive framework with increased performance for the use in cognitive systems for vehicle guidance automation tasks. It uses a combination of several existing graph transformation algorithms and techniques. We show, that for our approach, the incremental rule matching that we propose yields a performance gain over the non-incremental algorithm and a large increase over the existing generic cognitive framework COSA for a typical application.

Designing an Interactive Local and Global Decision Support System for Aircraft Carrier Deck Scheduling

In the near future, unmanned aerial vehicles will become part of the naval aircraft carrier operating environment. This will add significant complexity to an already highly constrained and dangerous environment. The move towards a shared manned-unmanned environment with an increasing operational tempo in a reduced manning environment will mean more automation is needed in the planning and scheduling of aircraft, ground vehicles, and crew in these complex environments. However, while automated planning algorithms are fast and able to handle large quantities of information in a short period of time, they are often brittle, unable to cope with changing conditions in highly dynamic environments. Recent research has shown that by allowing high-level interaction between human operators and automated planners, significant increases in overall mission performance can achieved. To this end, a user interface has been developed that allows a human decision maker managing aircraft carrier deck operations the ability to interact directly with a centralized planning algorithm for scheduling aircraft in flight and on the deck (both manned and unmanned), as well as ground vehicles and personnel. This Deck operations Course of Action Planner (DCAP) system leverages the experience and high-level, goal-directed behavior of the human decision maker in conjunction with a powerful automated planning algorithm to develop feasible, robust schedules. This article highlights the design features of DCAP and presents preliminary results from an evaluation designed to quantify the value added by layering in planning and scheduling algorithms into this complex decision process.

Human-System Interaction Analysis for Military Pilot Activity and Mental Workload Determination

We investigate the operationalization of mental workload (MWL) for adaptive pilot support. Common approaches have proven their benefits either for functional allocation in automation and crew station design, or in statistically grounded human-machine system evaluations. In this article, we argue why many established methods are not viable for a continuous, nonintrusive, context-rich, absolute rating of MWL. Instead, we suggest a novel threefold concept. Firstly, MWL depends on the given work objectives and the human operators tasks. Secondly, we understand MWL as induced by the current human activity and related mental resource demands. Thirdly, MWL influences task and activity related, observable human behavior patterns. We argue that each of these determinants increasingly refines the quantification of MWL. In this article, we present the current state of our concept and prototype implementation. The concept has consequences for the design of cognitive pilot associate systems in manned-unmanned systems in military aviation.

Measuring self-adaptive UAV operators' load-shedding strategies under high workload

This article focuses on the experimental identification of changes in human behaviour patterns of UAV-operators guiding multiple UAVs from a helicopter cockpit. These changes are based on self-regulation mechanisms of the operators to adapt to the current task and workload demands. Main objective of the use of these so called self-adaptive strategies is to avoid overload situations, and to retard exceeding capacity limits, to maintain overall acceptable performance as long as possible. Expressed by shedding and deferring tasks of lesser importance, or the relaxation of self-imposed criteria, these strategies lead to an observable change of human behaviour patterns, prior to grave performance decrements. This article describes a laboratory experiment utilising a virtual flight simulator to stimulate operators workload and observe their mitigation strategies by means of gaze detection and a detailed interaction monitoring. Using the observed behaviour changes in an assistant system as indicator for high workload situations of the operator, it shall be possible to support the operator prior the occurrence of errors.

Using cognitive automation for aircraft general systems management

A todays trend within the aircraft industry is towards a so-called More Electric Aircraft (MEA) architecture, using electric power as single type of energy to supply most of the systems on board of an air vehicle. Besides the various advantages, e.g. improved efficiency, reduced operating costs, etc., this will increase the complexity of the electrical system and therefore have an impact on the related Control and Monitoring System. Making use of new opportunities such as an efficient power management will even further increase the complexity of system management. Additional requirements regarding decision making and communication arise when used on board of uninhabited aerial vehicles (UAVs). The concept of cognitive automation and the Cognitive System Architecture developed on the Munich University of the Armed Forces (UBM) is well suited to address many of these challenges. In this paper we investigate the application of Cognitive Automation for an advanced power management within a MEA architecture. After the system related aspects of the new MEA architecture are presented, the theory of Cognitive Automation is briefly introduced. The method to achieve a centralised knowledge representation for the General Systems domain is presented and two different approaches to planning within this framework are discussed. Both approaches were implemented and verified using a simulation model of the general systems.

Human-UAV Co-operation Based on Artificial Cognition

In the future, Uninhabited Aerial Vehicles (UAVs) will be part of both civil and military aviation. This includes co-operative mission accomplishment of manned and unmanned assets with little manpower being available for UAV guidance. So, UAVs need to be able to accomplish tasks with a minimum of human intervention and possibly in co-operation with other UAVs or manned aircraft. This paper presents artificial cognition as approach to co-operative capabilities of UAVs. They are guided by so-called Artificial Cognitive Units (ACUs) being capable of goal-directed behavior on the basis of understanding the current situation. Prototype evaluation results show the capability of suchlike co-operative ACUs to yield human-like rationality and the ability to act as peers in a human-ACU team.