Sae Schatz

Tractable POMDP representations for intelligent tutoring systems

With Partially Observable Markov Decision Processes (POMDPs), Intelligent Tutoring Systems (ITSs) can model individual learners from limited evidence and plan ahead despite uncertainty. However, POMDPs need appropriate representations to become tractable in ITSs that model many learner features, such as mastery of individual skills or the presence of specific misconceptions. This article describes two POMDP representations—state queues and observation chains—that take advantage of ITS task properties and let POMDPs scale to represent over 100 independent learner features. A real-world military training problem is given as one example. A human study (n = 14) provides initial validation for the model construction. Finally, evaluating the experimental representations with simulated students helps predict their impact on ITS performance. The compressed representations can model a wide range of simulated problems with instructional efficacy equal to lossless representations. With improved tractability, POMDP ITSs can accommodate more numerous or more detailed learner states and inputs.

Adaptive Perceptual Training in a Virtual Environment

The United States military’s strategic position is evolving, and as a result, the Services are emphasizing the importance of sociocultural pattern recognition, sensemaking in ambiguous urban contexts, and understanding of patterns of life. In fact, military personnel at increasingly lower echelons are expected to possess these nuanced psychosocial perception and decision-making skills. To facilitate training of these complex competencies, the authors are developing a Virtual Observation Platform, an immersive virtual environment designed to adaptively train US Marine Corps personnel in sustained observation, sociocultural pattern recognition, anomaly detection, and other perceptual–cognitive skills. This paper briefly describes the purpose of the system and then covers its adaptive instructional tailoring in detail. The Platform’s adaptive features include information quality/quantity manipulation and instructional scaffolding in the form of communications from a virtual squad (i.e., peers) that are intrinsic to the narrative of the scenario.

Automatic Scenario Generation through Procedural Modeling for Scenario-Based Training

We discuss our current efforts at developing automatic scenario generation software. We begin by explaining the rationale, and then review successful previous efforts. We discuss the lessons-learned from the past work, and the conceptual pieces that are required to generate operationally-valid scenarios that support effective training. We then present the conceptual design of our scenario generation approach, which uses novel procedural modeling approaches to ensure operational and training requirements are adequately met.

Advanced Situated Tutors: Design, Philosophy, and a Review of Existing Systems:

“Situated tutors” combine intelligent, adaptive instructional technology with a simulated environment that allows trainees to explore the context, knowledge, applications, and social interactions inherent in the real-world equivalent. However, the situated tutor construct is, as yet, only superficially described. Thus, this paper seeks to add to the academic conceptualization of situated tutors by clearly defining these systems and their features. We go on to define “advanced situated tutors” as the most robust class of situated tutors, and then give examples of such systems.

ITS + SBT: A review of operational situated tutors.

Situated tutors combine features of intelligent tutoring systems with simulated environments. A construct definition for situated tutors was newly developed, and, in this article, we use it to classify an extensive review of operational instructional technologies in order to identify a robust set of situated tutors. We document 86 situated tutors, half of which directly support military training. Initial empirical evidence from these systems suggests that situated tutors may outperform traditional simulations or intelligent tutors. As additional evidence is gathered, we hypothesize that situated tutors will also demonstrate efficiency and effectiveness in cognitive readiness instruction.

Contrasting Cases: A Strategy for Advanced Learning Using Simulation-based Training

Scenario-based training (SBT) as a medium for facilitating higher order or advanced learning has not been exploited fully. This paper argues for the use of contrasting cases to facilitate advanced learning within practice environments, where advanced learning is characterized by well differentiated and organized knowledge that can be generalized and abstracted. The rationale and guidance for incorporating contrasting cases into SBT is provided. A learning framework is leveraged to further describe how the method of contrasting cases can be adapted to address the challenges of learners at different stages of skill acquisition.

Advancing the Science of Training in Simulation-Based Training

Recent changes in battlefield dynamics are driving the need to train higher-order cognitive skills such as planning and decision making over the more common focus of declarative knowledge and procedural skills. These changes have therefore created a need for changes in military training processes. Some questions that arise from this need include how to ensure that such higher-order skills and knowledge are meaningfully retained in long-term memory, how to support military personnel in applying the proper knowledge and skills under stressful operational conditions, and what pedagogical strategies can be applied to best train these skills. To deal with these issues, the military has pursued a number of research and development projects investigating scenario-based training in simulators. The notion is that appropriately designed scenarios can help the learner connect task elements in such a way that knowledge is better integrated, better retained, and more easily applied even under demanding operational contexts. While scenario-based training (SBT) remains one of the more popular approaches for teaching complex skills, the instantiation of complementary learning strategies within this context has not been fully explored. Further, most simulators do not support the SBT instructional approach; instead human instructors must carry out the steps of SBT themselves. Additionally, developing SBT scenarios and other instructional content remains challenging and these processes, too, remain under-supported. Thus, extending SBT to include complementary learning strategies as well as automating certain components of SBT would significantly improve the effectiveness and ease the implementation of this method. As the military and industrial communities continue to design and develop scenario-based simulation systems, it seems prudent to investigate technologies and pedagogy that can improve their impact. In this symposium we describe a multidisciplinary effort, funded by the Office of Naval Research, designed to address the multi-faceted needs of improving scenario-based simulations to meet the needs of the modern warfighter. We take a two-pronged approach to this effort involving complementary technological and pedagogical research thrusts. Specifically, we first leverage findings from the cognitive and learning sciences regarding the learning process and development of expertise in order to implement them within the context of SBT. Second, we utilize developments in the computational and information sciences to examine how scenario generation and computational models can support the automation of SBT. Finally, we conclude by illustrating how these projects are closely coupled and monitored using methods developed for impact assessment. In this overview we briefly describe these approaches and highlight their contribution to the extant science of SBT and their relevance to military training.

Augmented Cognition and Training in the Laboratory: DVTE System Validation

As the modern workplace becomes more complex, the training community needs to develop new strategies to continute to create a competent workforce. The emerging field of augmented cognition may be able to contribute greatly to increasing training capability through the use of neurophysiological measures that support real-time performance and can be used to satisfy some of the requirements of an automated intelligent tutoring system. The first step to building this system is to design and validate a testbed that can be used with future efforts to target effective mitigation strategies with learning efficiency. In this study, participants watched a computerized instructional presentation and then engaged in a practice CFF scenario in the simulator. When finished, each participant was assigned to either a low or high task load test scenario. In both, the goal was to destroy five enemy tanks. Some participants were also asked to simultaneously execute a secondary radar monitoring task. Both the National Aeronautics and Space Administrations Task Load Index (NASA-TLX) and the MRQ were used to assess the validity of the testbed. Results from both measures indicate that a significant difference exists between the two levels of workload and further, we can distinguish between the subscales within each measurement tool. Thus, the overarching goal of this study was achieved. High and low workload scenarios were created and validated. Ultimatly, they will be used as a testbed of scenarios to address the questions surrounding the use of neurophysiological equipment to impact individual learning patterns.

A Review of Military Predeployment Stress Tolerance Training

Stress-induced psychological disorders and related attrition are major concerns for the U.S. Military. In fact, the Marine Corps has identified “mental toughness,” a construct including psychological resilience, among its critical competencies. In response to the psychological sequelae of exposure to stressful and potentially traumatic environments, the Services have recently implemented programs designed to promote stress tolerance. However, empirical evidence to support the efficacy of these training efforts is sorely lacking. As a result, these programs are often disjoint, cross-sectional, and delivered haphazardly. As a preliminary step toward addressing these issues, this outlines the efforts currently in use by the armed services, as well as those recommended by related communities. We present the potential shortcomings of these contemporary efforts, identify best-of-breed approaches and outcomes, and recommend next steps to improve resilience training efforts.

Scenario-Based Training: Scenario Complexity

This paper presents and extends the definition of scenario complexity in support of the Next-generation Expeditionary Warfare Intelligent Training (NEW-IT) project. First, key concepts are defined with a focus on two specific characteristics: task complexity (Wood, 1986) and task framework (Campbell, 1988). The authors then propose the inclusion of cognitive task elements to the traditional task complexity formulation as an integral extension when designing scenarios for higher-order thinking skills. In addition, the authors introduce the variable of cognitive context moderators to the developing equation. Demonstration of the proposed formulation is then presented with three examples of the instantiation. Finally, operationalization and future direction for scenario-based training are discussed.