Michael C. Dorneich

Toward a Characterization of Adaptive Systems: A Framework for Researchers and System Designers

Objective: This article presents a systematic framework characterizing adaptive systems. Background: Adaptive systems are those that can appropriately modify their behavior to fit the current context. This concept is appealing because it offers the possibility of creating computer assistants that behave like good human assistants who can provide what is needed without being asked. However, the majority of adaptive systems have been experimental rather than practical because of the technical challenges in accurately perceiving and interpreting users’ current cognitive state; integrating cognitive state, environment, and task information; and using it to predict users’ current needs. The authors anticipate that recent developments in neurological and physiological sensors to identify users’ cognitive state will increase interest in adaptive systems research and practice over the next few years. Method: To inform future efforts in adaptive sys-tems, this work provides an organizing framework for characterizing adaptive systems, identifying consider-ations and implications, and suggesting future research issues. Results: A two-part framework is presented that (a) categorizes ways in which adaptive systems can modify their behavior and (b) characterizes trigger mechanisms through which adaptive systems can sense the current situation and decide how to adapt. Conclusion: The framework provided in this article provides a tool for organizing and informing past, present, and future research and development efforts in adaptive systems.

GLOBAL OPTIMIZATION ALGORITHMS FOR CHIP LAYOUT AND COMPACTION

The package planning (chip layout and compaction) problem can be stated in terms of an optimization problem. The goal is to find the relative placement and shapes of the chips in a way that minimizes the total chip area subject to linear and nonlinear constraints. The constraints arise from geometric design rules, distance and connectivity requirements between various components, area and communication costs and other designer-specified requirements. The problem has been addressed in various settings. It is of unusual computational difficulty due to the nonconvexities- involved. This paper presents a new mixed-integer nonlinear programming formulation for simultaneous chip layout and two-dimensional compaction. Global optimization algorithms are developed for this model as well as for an existing formulation for the chip compaction problem. These algorithms are implemented with the global optimization software BARON and illustrated by solving several example problems.

Considering Etiquette in the Design of an Adaptive System

In this article, the authors empirically assess the costs and benefits of designing an adaptive system to follow social conventions regarding the appropriateness of interruptions. Interruption management is one area within the larger topic of automation etiquette. The authors tested these concepts in an outdoor environment using the Communications Scheduler, a wearable adaptive system that classifies users’ cognitive state via brain and heart sensors and adapts its interactions. Designed to help dismounted soldiers, it manages communications in much the same way as a good administrative assistant. Depending on a combination of message priority, user workload, and system state, it decides whether to interrupt the user’s current tasks. The system supports decision makers in two innovative ways: It reliably measures a mobile user’s cognitive workload to adapt its behavior, and it implements rules of etiquette adapted from human-human interactions to improve human-computer interactions. Results indicate costs and benefits to both interrupting and refraining from interrupting. When users were overloaded, primary task performance was improved by managing interruptions. However, overall situation awareness on secondary tasks suffered. This work empirically quantifies costs and benefits of “appropriate” interruption behaviors, demonstrating the value of designing adaptive agents that follow social conventions for interactions with humans.

Rapid image analysis using neural signals

The problem of extracting information from large collections of imagery is a challenge with few good solutions. Computers typically cannot interpret imagery as effectively as humans can, and manual analysis tools are slow. The research reported here explores the feasibility of speeding up manual image analysis by tapping into split second perceptual judgments using electroencephalograph sensors. Experimental results show that a combination of neurophysiological signals and overt physical responses--detected while a user views imagery in high speed bursts of approximately 10 images per second--provide a basis for detecting targets within large image sets. Results show an approximately six-fold, statistically significant, reduction in the time required to detect targets at high accuracy levels compared to conventional broad-area image analysis.

Cognitive State Estimation Based on EEG for Augmented Cognition

Augmented cognition is an emerging concept that aims to enhance user performance and cognitive capabilities on the basis of adaptive assistance. An integral part of such systems is the automatic assessment of the instantaneous cognitive state of the user. This paper describes an automatic cognitive state estimation methodology based on the use of EEG measurements with ambulatory users. The required robustness in this context is achieved through the use of a mutual information based dimensionality reduction approach in conjunction with a committee of classifiers, and median filter outlier rejection element. We present classification results associated with cognitive tasks performed in mobile and stationary modalities

Supporting Real-time Cognitive State Classification on a Mobile Individual

The effectiveness of neurophysiologically triggered adaptive systems hinges on reliable and effective signal processing and cognitive state classification. Although this presents a difficult technical challenge in any context, these concerns are particularly pronounced in a system designed for mobile contexts. This paper describes a neurophysiologically derived cognitive state classification approach designed for ambulatory task contexts. We highlight signal processing and classification components that render the electroencephalogram (EEG) -based cognitive state estimation system robust to noise. Field assessments show classification performance that exceeds 70% for all participants in a context that many have regarded as intractable for cognitive state classification using EEG.

Neurophysiologically driven image triage: a pilot study

Effective analysis of complex imagery is a vital aspect of important domains such as intelligence image analysis. As technological developments lower the cost of gathering and storing imagery, the cost of searching through large image sets for important information has been growing substantially. This paper demonstrates the feasibility of using neurophysiological signals associated with early perceptual processing to identify critical information within large image sets efficiently. Brain signals called evoked response potentials, detected in conjunction with rapid serial presentation of images, show promise as a human computer interaction modality for screening high volumes of imagery accurately and efficiently.

A system design framework-driven implementation of a learning collaboratory

This paper describes a design process to support the development of a learning collaboratory, a distributed, computer-based, virtual space for learning and work. A learning collaboratory, as a distributed distance learning environment, offers great opportunities to expand the way people teach and learn and to broaden educational opportunities to an ever increasing range of learners. The challenge is to design distance learning technologies that engender meaningful learning experiences that take full advantage of the power of computer-mediated communication to support innovative learner-centered and collaborative interactions between students, teachers, subject experts, and resources. First, the paper describes the learning collaboratory design framework (LUCIDIFY), a design process that integrates methods and concepts from cognitive systems engineering, theories of learning and instruction, distributed computing, and computer-supported collaborative learning to guide the principled design of learning collaboratories. Next, the paper describes how LUCIDIFY was used in the design and implementation of the collaborative learning environment for operational systems (CLEOS), a learning collaboratory for teachers, students, and practitioners in the physical sciences. CLEOS features two virtual instrument tutorials, an asynchronous messaging system, a project-based design and management application, and a collaborative multi-user domain infrastructure.

A joint human-automation cognitive system to support rapid decision-making in hostile environments

Honeywell has designed a joint human-computer cognitive system to support rapid decision making demands of dismounted soldiers. In highly networked environments the sheer magnitude of communication amid multiple tasks could overwhelm individual soldiers. Key cognitive bottlenecks constrain information flow and the performance of decision-making, especially under stress. The adaptive decision-support system mitigates non-optimal human performance via automation when the system detects a breakdown in the humans cognitive state. The humans cognitive state is assessed in real-time via a suite of neuro-physiological and physiological sensors. Adaptive mitigation strategies can include task management, optimizing information presentation via modality management, task sharing, and task loading. Mitigations are designed with consideration for both the costs and benefits of intermittent augmentation. The paper describes the system development and evolution, explorations of usable cognitive mitigation strategies, and four evaluations that show adaptive automaton can effectively, mitigate human decision-making performance at extremes (overload and underload) of workload.

Closing the Loop of an Adaptive System with Cognitive State

This paper describes an adaptive system that “closes the loop” by utilizing a real-time, directly sensed measure of cognitive state of the human operator. The Honeywell Augmented Cognition team has developed a Closed Loop Integrated Prototype (CLIP) of a Communications Scheduler, for application to the U.S. Armys Future Force Warrior (FFW) program. It is expected that in a highly networked environment the sheer magnitude of communication traffic could overwhelm the individual soldier. The CLIP exploits real-time neurophysiological and physiological measurements of the human operator in order to create a cognitive state profile, which is used to augment the work environment to improve human-automation joint performance. An experiment showed that the Communications Scheduler enabled higher situation awareness and message comprehension in high workload conditions. Based solely on cognitive state, the system inferred a subjects message comprehension and repeated unattended messages in the majority of cases, without yielding an unacceptably high false alarm rate.