Denise Nicholson

A Paradigm Shift in Interactive Computing: Deriving Multimodal Design Principles from Behavioral and Neurological Foundations

As technology advances, systems are increasingly able to provide more information than a human operator can process accurately. Thus, a challenge for designers is to create interfaces that allow operators to process the optimal amount of data. It is herein proposed that this may be accomplished by creating multimodal display systems that augment or switch modalities to maximize user information processing. Such a system would ultimately be informed by a users neurophysiological state. As a first step toward that goal, relevant literature is reviewed and a set of preliminary design guidelines for multimodal information systems is suggested.

Cognitive Load Theory vs. Constructivist Approaches: Which Best Leads to Efficient, Deep Learning?

Computer-assisted learning, in the form of simulation-based training, is heavily focused upon by the military. Because computer-based learning offers highly portable, reusable, and cost-efficient training options, the military has dedicated significant resources to the investigation of instructional strategies that improve learning efficiency within this environment. In order to identify efficient instructional strategies, this paper investigates the two major learning theories that dominate the recent literature on optimizing knowledge acquisition: cognitive load theory (CLT) and constructivism. According to CLT, instructional guidance that promotes efficient learning is most beneficial. Constructivist approaches, in contrast, emphasize the importance of developing a conceptual understanding of the learning material. Supporters of these theories have debated the merits and shortcomings of both positions. However, in the absence of consensus, instructional designers lack a well-defined model for training complex skills in a rapid, efficient manner. The current study investigates the relative utility of CLT and constructivist-based approaches for teaching complex skills using a military command and control task. Findings suggest that the acquisition of procedural, declarative, and conceptual knowledge, as well as decision-making skills, did not differ as a function of the type of instruction used. However, integrated knowledge was slightly better retained by the group provided with CLT-based instruction. These results are contrary to our expectation that constructivist approaches, which focus on the development and integration of information, would yield better performance in an applied problem-based environment. Thus, while contemporary researchers continue to defend the use of constructivist strategies for teaching, our research supports earlier findings that question the utility, efficiency, and impact of these strategies in applied domains.

Comparison of multiple physiological sensors to classify operator state in adaptive automation systems

Automating tasks alleviates operator resources to be delegated to other demands, but the cost is often situation awareness. In contrast, complete manual control of a system opens the door for greater human error. Therefore, an ideal situation would require the development of an adaptive system in which automation can be triggered based on performance of a particular task, time spent on the task, or perhaps physiological response. The latter pertains to the goal for this particular study. Electroencephalogram (EEG), electrocardiogram (ECG), and eye tracking measures were recorded during six multi-tasking scenarios to assess if any one single measure is best suited for future implementation as an automation invocation. EEG showed the greatest potential for that purpose.

Adaptive Perceptual Training in a Virtual Environment

The United States military’s strategic position is evolving, and as a result, the Services are emphasizing the importance of sociocultural pattern recognition, sensemaking in ambiguous urban contexts, and understanding of patterns of life. In fact, military personnel at increasingly lower echelons are expected to possess these nuanced psychosocial perception and decision-making skills. To facilitate training of these complex competencies, the authors are developing a Virtual Observation Platform, an immersive virtual environment designed to adaptively train US Marine Corps personnel in sustained observation, sociocultural pattern recognition, anomaly detection, and other perceptual–cognitive skills. This paper briefly describes the purpose of the system and then covers its adaptive instructional tailoring in detail. The Platform’s adaptive features include information quality/quantity manipulation and instructional scaffolding in the form of communications from a virtual squad (i.e., peers) that are intrinsic to the narrative of the scenario.

Automatic Scenario Generation through Procedural Modeling for Scenario-Based Training

We discuss our current efforts at developing automatic scenario generation software. We begin by explaining the rationale, and then review successful previous efforts. We discuss the lessons-learned from the past work, and the conceptual pieces that are required to generate operationally-valid scenarios that support effective training. We then present the conceptual design of our scenario generation approach, which uses novel procedural modeling approaches to ensure operational and training requirements are adequately met.

Advanced Situated Tutors: Design, Philosophy, and a Review of Existing Systems:

“Situated tutors” combine intelligent, adaptive instructional technology with a simulated environment that allows trainees to explore the context, knowledge, applications, and social interactions inherent in the real-world equivalent. However, the situated tutor construct is, as yet, only superficially described. Thus, this paper seeks to add to the academic conceptualization of situated tutors by clearly defining these systems and their features. We go on to define “advanced situated tutors” as the most robust class of situated tutors, and then give examples of such systems.

An adaptive instructional architecture for training and education

Office of Naval Research (ONR) initiatives such as Human Performance Training and Education (HPT&E) as well as Virtual Technologies and Environments (VIRTE) have primarily focused on developing the strategies and technologies for creating multimodal reality or simulation based content. Resulting state-of-the-art training and education prototype simulators still rely heavily on instructors to interpret performance data, and adapt instruction via scenario generation, mitigations, feedback and after action review tools. Further research is required to fully close the loop and provide automated, adaptive instruction in these learning environments. To meet this goal, an ONR funded initiative focusing on the Training and Education arm of the HPT&E program will address the processes and components required to deliver these capabilities in the form of an Adaptive Instructional Architecture (AIA). An overview of the AIA as it applies to Marine Corps Warfighter training protocols is given as well as the theoretical foundations supporting it.

Recommended roles for uninhabited team members within mixed-initiative combat teams

Trust in automation is a well-researched topic that is particularly important when planning mixed initiative interaction. When working with teams comprised of both human and non-human team members, the amount of trust the operator places in the automation often determines which parts of the interaction can be automated and the optimal level of automation. The mixed-initiative community has created numerous systems that leverage trust in automation, but results have been inconclusive. After examining the primary factors that impact trust in automated systems, we make several recommendations regarding the assignment of roles for human and non-human mixed-initiative team members.

Increasing Efficiency in Military Learning: Theoretical Considerations and Practical Applications

Given the fundamental importance of higher-order cognitive skills for military personnel, increasing learning efficiency during training is paramount. The current article expands upon the state-based information-loss processing model, a comprehensive framework elucidating the processes involved in acquiring higher-order cognitive skills, to enumerate best practices for military training. Emphasis is placed on identifying empirically supported, state-of-the-art learning efficiency strategies and methodologies to address points of information loss throughout the learning process. Implications and pragmatic recommendations for simulation-based military training are discussed.

Assessing Cognitive State with Multiple Physiological Measures: A Modular Approach

The purpose of this effort is to introduce a novel approach which can be used to determine how multiple minimally intrusive physiological sensors can be used together and validly applied to areas such as Augmented Cognition and Neuroergonomics. While researchers in these fields have established the utility of many physiological measures for informing when to adapt systems, the use of such measures together remains limited. Specifically, this effort will provide a contextual explanation of cognitive state, workload, and the measurement of both; provide a brief discussion on several relatively noninvasive physiological measures; explore what a modular cognitive state gauge should consist of; and finally, propose a framework based on the previous items that can be used to determine the interactions of the various measures in relation to the change of cognitive state.