Crystal S. Maraj

Exploring the Impact of Simulator Sickness on the Virtual World Experience

This research initiative seeks to understand training effectiveness for Virtual World (VW) technologies. The advancement of VW training platforms in both academia and industry demonstrates the need to investigate the effects of simulator sickness (SS) on the VW experience. This paper uses the Military Open Simulator Enterprise Strategy (MOSES) VW platform to train 32 Reserve Officers’ Training Corps Cadets from the University of Central Florida on a room clearing task. A between-subjects design was conducted with an emphasis on the VW training condition. The data collected included both individual and collective performance as well as perceptual data (SS, Presence, Flow, and Engagement). Data analysis comprised of a series of Pearson product correlation coefficient for understanding the relationship between SS and Presence, Flow, and Engagement in addition to explaining performance outcomes. The results found moderate to strong, negative correlations between SS and Presence, SS and Flow, and SS and Engagement. The results reveal that SS interrupts presence during the VW training which can lead to negative training transfer. Alternatively, the increase in presence, flow, and engagement associated with a decrease in SS may be linked to motivation which is essential to effective training. Finally, this paper discusses limitations related to VW research (e.g., internal and external validity, expertise level, etc.) but also provides a basic foundation from which SS research may enhance VW training.

Impact of Instructional Strategies on Motivation and Engagement for Simulation-Based Training of Robot-Aided ISR Tasks

The U.S. Army projects a considerable increase in the number of operational Unmanned Ground Systems (UGS) within the next ten years. There is a need to enhance UGS capabilities to support remote Intelligence, Surveillance, Reconnaissance (ISR) operations involving the identification of High-Value Individuals (HVI). Emerging UGS capability requirements will inevitably result in new or revised training requirements. The U.S. Army identifies Simulation-Based Training (SBT) as a required training platform for robot-aided ISR tasks utilizing UGSs. In order to implement an effective SBT system, there are several factors to consider related to training systems design and trainee needs. Factors addressed in this study include the selection of effective SBT instructional strategies and the impact on trainee motivation and engagement. Results from this study contribute to design and future research recommendations regarding SBT for robot-aided ISR tasks.

Assessing Engagement in Simulation-Based Training Systems for Virtual Kinesic Cue Detection Training

Combat Profiling techniques strengthen a Warfighter’s ability to quickly react to situations within the operational environment based upon observable behavioral identifiers. One significant domain-specific skill researched is kinesics, or the study of body language. A Warfighter’s ability to distinguish kinesic cues can greatly aid in the detection of possible threatening activities or individuals with harmful intent. This paper describes a research effort assessing the effectiveness of kinesic cue depiction within Simulation-Based Training (SBT) systems and the impact of engagement levels upon trainee performance. For this experiment, live training content served as the foundation for scenarios generated using Bohemia Interactive’s Virtual Battlespace 2 (VBS2). Training content was presented on a standard desktop computer or within a physically immersive Virtual Environment (VE). Results suggest that the utilization of a highly immersive VE is not critical to achieve optimal performance during familiarization training of kinesic cue detection. While there was not a significant difference in engagement between conditions, the data showed evidence to suggest decreased levels of engagement by participants using the immersive VE. Further analysis revealed that temporal dissociation, which was significantly lower in the immersive VE condition, was a predictor of simulation engagement. In one respect, this indicates that standard desktop systems are suited for transitioning existing kinesic familiarization training content from the classroom to a personal computer. However, interpretation of the results requires operational context that suggests the capabilities of high-fidelity immersive VEs are not fully utilized by existing training methodologies. Thus, this research serves as an illustration of technology advancements compelling the SBT community to evolve training methods in order to fully benefit from emerging technologies.

Contrasting Instructional Strategies Suited to a Detection Task: Examining Differences in Subjective Workload

Soldiers benefit from the ability to detect threats conveyed via human kinesic cues, or non-verbal body movements. Simulation-Based Training (SBT) supplies an avenue to improve kinesic cue detection performance. Instructional strategies in SBT are designed to enhance performance outcomes, and are tied to workload in terms of training effectiveness. Three instructional strategies were analyzed in a between-subjects design for their degree of perceived workload: Highlighting, Massed Exposure, and Kim’s game. Workload included subjective mental and global demand subscales from the NASA-Task Load Index (NASA-TLX). A multivariate analysis of variance showed that Kim’s game contained the highest mental and global demands; Highlighting produced the lowest mental and global demands. The differences in demands suggest rationales for strategy placement, in terms of combinations (e.g., layers or progressions) and other applications (e.g., air traffic control, medical diagnosis, and After-Action Reviews). The strategies’ workload differences are also traced to differences in attention and working memory.

Impact of Instructional Strategies on Workload, Stress, and Flow in Simulation-Based Training for Behavior Cue Analysis

The U.S. Army desires to improve Intelligence, Surveillance, Reconnaissance (ISR) abilities by incorporating Unmanned Ground Systems (UGS) to aid in the identification of High Value Individuals (HVI) through the analysis of human behavior cues from safer distances. This requires analysts to employ perceptual skills indirectly via UGS video surveillance displays and will also require training platforms tailored to address the perceptual skill needs of these robot-aided ISR tasks. The U.S. Army identifies Simulation-Based Training (SBT) as a necessary training medium for UGS technologies. Instructional strategies that may increase the effectiveness of SBT for robot-aided ISR tasks include Highlighting and Massed Exposure. This study compared the impact of each strategy on trainee workload, stress, and flow during SBT for a behavior cue analysis task. Ultimately, the goal of this research effort is to provide instructional design recommendations that will improve SBT development to support effective training for emerging UGS capabilities.

Assessment of Kim’s Game Strategy for Behavior Cue Detection: Engagement, Flow, & Performance Aspects

Psychological constructs, such as engagement and flow, can be used to determine an individual’s involvement in a task and predict levels of performance during Simulation-Based Training (SBT) in military operations. This experiment used a virtual form of Kim’s game (an observational training game that includes memorization of objects and later recall), to improve pattern recognition and behavior cue detection, during SBT. The purpose of this experiment was to assess participant engagement and flow between two conditions, Kim’s game vs. the control. Seventy-five participants were randomly assigned to either condition, and completed a pre-test, training vignette, post-test, and multiple questionnaires which assessed the individuals’ levels of engagement and flow. Experimental results show the control group reported higher levels of both engagement and flow in all subscales, and flow as a higher predictor of performance than engagement. This paper examines plausible explanations why the engagement questionnaire did not assess differences in performance. The lack of statistically significant findings may be a result of the engagement survey questions and the type of task (i.e., discrete or continuous). Finally, this paper provides future recommendations for examining the role of engagement and flow for simulation-based behavior cue detection training.

Preliminary Review of a Virtual World Usability Questionnaire

Improving performance through training in virtual environments has led to identifying the best methods associated with enhancing human-computer interaction. This paper provides a description of a usability questionnaire for Virtual World training, by focusing on experiential U.S. Army Warrior Leader Course tasks performed using an input desktop device (i.e., a mouse and keyboard). The usability questionnaire was developed by integrating established usability subscales. A Likert scale for both user experience (for game genre and controller type) and usability levels were implemented, along with free response questions, to gain insight into the controller interface(s). Descriptive statistics and determination of internal consistency (using Cronbach’s alpha) are reviewed for establishing the reliability and validity of the novel questionnaire. Proposed changes to the questionnaire include removal of superfluous items and consolidation of subscales. Tradeoffs for types of controllers are discussed, in light of the results.

Effects of Visual Fidelity on Biometric Cue Detection in Virtual Combat Profiling Training

Combat Profiling involves observation of humans and the environment to identify behavioral anomalies signifying the presence of a potential threat. Desires to expand accessibility to Combat Profiling training motivate the training community to investigate Virtual Environments (VEs). VE design recommendations will benefit efforts to translate Combat Profiling training methods to virtual platforms. Visual aspects of virtual environments may significantly impact observational and perceptual training objectives. This experiment compared the effects of high and low fidelity virtual characters for biometric cue detection training on participant performance and perceptions. Results suggest that high fidelity virtual characters promote positive training perceptions and self-efficacy, but do not significantly impact overall performance.

Investigation of the Impact of Feedback on Decision Accuracy and Reaction Time in a Perceptual Training Task

This study investigated the impact of feedback interventions on a perceptual aviation training task. Previous research on feedback suggests that knowledge of results and conceptual feedback can facilitate learning in certain contexts. Using an aviation discrimination training module, we sought to examine this influence in perceptual training and collected data from 52 volunteer participants. Initial results suggested that there was no noticeable difference between feedback and non-feedback conditions with respect to post-training performance. However, an interesting finding stemming from the average reaction times during training was that knowledge of results may alter response strategies in training: Whereas the no-feedback and conceptual feedback groups required significantly more time for their responses during training than either in the pre- or post-test (suggesting that they needed time to “think through” their responses in the training), the knowledge-of-results group was not only faster than the other groups in training, but its respondents also did not show the increase in response times during training that was so characteristic of the other two conditions. We concluded that this suggested that knowledge of results allowed participants to respond quickly and without the need of time to “think through” their responses. Based on the outcome of this study, we discuss the training implications for training perceptual tasks that rely on accurate and timely action.

A Usability Review of the Learning Master Serious Game in Support of the US Army Jumpmaster’s Course

The use of serious games for training can promote engagement and encourage curiosity. While there is controversy over how to assess the effectiveness of using serious games for training, research efforts have linked instructional effectiveness to instructional objectives and game characteristics. This paper describes these components in relations to a serious game for students in the United States Army Jumpmaster course. The authors collected data on usability through user feedback from surveys after the use of a serious game called Learning Master. From the data collected, users perceived the serious game to be simple to use, function appropriately, and have an intuitive design. A majority of the users expressed confidence in using the system. The user responses called for improvements in the visual display and simplified access to the application. The design team plans to use this information to enhance and finalize the serious game for future testing.