Eileen M. Smith

Mixed reality in education, entertainment, and training

Transferring research from the laboratory to mainstream applications requires the convergence of people, knowledge, and conventions from divergent disciplines. Solutions involve more than combining functional requirements and creative novelty. To transform technical capabilities of emerging mixed reality (MR) technology into the mainstream involves the integration and evolution of unproven systems. For example, real-world applications require complex scenarios (a content issue) involving an efficient iterative pipeline (a production issue) and driving the design of a story engine (a technical issue) that provides an adaptive experience with an after-action review process (a business issue). This article describes how a multi-disciplinary research team transformed core MR technology and methods into diverse urban terrain applications. These applications are used for military training and situational awareness, as well as for community learning to significantly increase the entertainment, educational, and satisfaction levels of existing experiences in public venues.

Human Experience Modeler: Context-Driven Cognitive Retraining to Facilitate Transfer of Learning

We describe a cognitive rehabilitation mixed-reality system that allows therapists to explore natural cuing, contextualization, and theoretical aspects of cognitive retraining, including transfer of training. The Human Experience Modeler (HEM) mixed-reality environment allows for a contextualized learning experience with the advantages of controlled stimuli, experience capture and feedback that would not be feasible in a traditional rehabilitation setting. A pilot study for testing the integrated components of the HEM is discussed where the participant presents with working memory impairments due to an aneurysm.

Assessing Virtual Rehabilitation Design with Biophysiological Metrics

Efficacy of virtual rehabilitation applications is typically demonstrated by pre and post comparisons of observable behavioral metrics. These behaviors can be monitored via devices such as trackers or video capture and more traditional error rate metrics. However, monitoring the patient¿s emotional and cognitive changes during virtual rehabilitation may better guide the rehab process as well as the design of the rehab scenario. We explored the use of biophysiological metrics (EEG, GSR, and Respiration) in the design of a virtual restaurant for the purpose of engaging persons who stutter in verbal interactions during an everyday experience. The EEG results showed that participants experienced higher engagement in the virtual restaurant. Although respiration and GSR metrics differed for each participant, they correlated well with stressors presented in the scenario. The work supports the use of biophysiological measures as an objective means of assessing virtual rehabilitation protocols.

Working the Modes: Understanding the Value of Multiple Modalities of Technologies for Learning and Training Success

Technology for learning has a great potential to decrease training time, as well as impart complex knowledge to the learner. However, one technology may not provide the complete learning experience. We discuss this issue using a fielded fire rescue incident command simulation-based training. Of first importance is properly defining the training material, and then assessing the efficacy of the training through scenario-based critique. The immersive nature of the incident command simulation allowed learners of all ages and backgrounds to experience the realism of a fire command post. Newer immersive technologies are discussed that will support transfer of training, as well as provide seamless integration into real world settings. Finally, we advocate for the development of direct brain measures of the learning process within operational environments. In this way, instructional design becomes a true brain-based approach and selecting the supporting technology for learning delivery is more exact for the learning purpose.

Experiential learning with Mixed and Augmented Reality

Today’s world is a thrilling mixture of physical and virtual spaces, from the intimate conversation with a friend at the grocery store or the gym, to tweeting updates and Facebook entries to our friends around the globe. All around us, we are thirsting to be connected with our local and global world. When we explore things that interest us, emerging technology has given us the ability to delve much deeper into the topics if we desire. Mixed and Augmented Reality bring a unique contextualization opportunity for learning experiences the ability to customize the learning journey for each learner, on each topic, for each intended impact. That self-gathering of important information sets a personal context to our learning. Usercreated content in the 21st century interactive media is quite capable of allowing learners to create long-term learning records what each person considers important on their learning journey. It might be a video found online, or a digital record of a trip to a community garden, or an exploration into the types of sea turtles found off the Atlantic coast. How can we drive innovation with Mixed and Augmented Reality (MR/AR) to encourage learning? How do we put a life supply of learning for all ages into finite spaces such as museums and community learning centers? How do we evaluate the effectiveness of this learning? In this workshop we look to explore how to capture the imagination of the public. Speakers will each present a compelling picture of one aspect of MR/AR and its real-world impact on learning, followed by roundtable discussions on those aspects. This workshop is intended to address learning applications of Mixed Reality for K-12, higher education and professional and continuing education. This ranges from self-directed learning in the home, to specialized learning with MR simulation tools, to free-choice learning in museums, over the Internet, and in diverse education and entertainment venues throughout society.

Evaluating the Potential of Cognitive Rehabilitation with Mixed Reality

We describe the development and use of a mixed reality (MR) testbed to evaluate potential scenarios that may alleviate performance deficits in subjects who may be experiencing cognitive deficiencies, such as posttraumatic stress disorder (PTSD). The system blends real world sensory data with synthetic enhancements in the visual and aural domains. It captures user actions (movement, view direction, environment interaction, and task performance) and psychophysical states (engagement, workload, and skin conductivity) during an MR-enabled experience in order to determine task performance in the context of a variety of stimuli (visual and aural distracters in time-constrained activities). The goal is to discover triggers that affect stress levels and task performance in order to develop individualized plans for personal improvement.