Mina C. Johnson-Glenberg

Emboldened by Embodiment: Six Precepts for Research on Embodied Learning and Mixed Reality

The authors describe an emerging paradigm of educational research that pairs theories of embodied learning with a class of immersive technologies referred to as mixed reality (MR). MR environments merge the digital with the physical, where, for example, students can use their bodies to simulate an orbit around a virtual planet. Recent research supports the idea that body activity can be an important catalyst for generating learning, and new technologies are being developed that use natural human physicality and gesture as input. However, existing research on embodied learning technologies has been disparate, driven largely by specific technical innovations and constraints, and often lacking a clear focus on establishing their efficacy in educational contexts. On the basis of the unique characteristics of these technologies and on their own experiences conducting research in this area, the authors put forth six precepts for embodied learning technology researchers that pertain to the rationale, design, and execution of empirical studies.

Semi-virtual Embodied Learning-Real World STEM Assessment

This chapter presents several results from a multiyear research endeavor on embodiment in gaming and learning. A trans-disciplinary group at Arizona State University has designed an innovative learning environment that allows the learner’s body to move freely in space while interacting with dynamic visual and sonic media. This semi-virtual interface/environment is called SMALLab (Situated Multimedia Arts Learning Laboratory). The environment relies on 3D object tracking, real time graphics, and surround-sound to enhance embodied learning. Our hypothesis is that optimal learning and retention will occur when learning is embodied and multiple modalities are incorporated during the act of learning. In addition, we have created game-like scenarios that are collaborative and, where appropriate, incorporate the constructs of competition and low-stakes formative, stealth assessment to increase engagement, knowledge construction, and metrics on student learning.

Fragile X syndrome: Neural network models of sequencing and memory

A comparative framework of memory processes in males with fragile X syndrome (FXS) and Typically Developing (TYP) mental age-match children is presented. Results indicate a divergence in sequencing skills, such that males with FXS recall sequences similarly to TYP children around five and a half years of age, but eth males with FXS recall significantly worse when compared to TYP children around seven and a half years of age. Performance on one working memory measure, an n-back card task, is modeled with a neural network. To date, no network models explicate the sequencing and memory processes in those with FXS. Noise was added to various levels (weight matrices) in the FXS model and outputs approximated human FXS performance. Three models were compared: 1) FXS; 2) younger mental age-TYP matches; and 3) older reading level-TYP matches. Modeling can help to reify conceptualizations of deficits and to guide in the creation of more valid, science-based remediations. The FXS model suggests that the levels of phonological representation and sequencing in memory are candidates for targeted therapies in males with FXS.

Effects of embodied learning and digital platform on the retention of physics content: Centripetal force

Embodiment theory proposes that knowledge is grounded in sensorimotor systems, and that learning can be facilitated to the extent that lessons can be mapped to these systems. This study with 109 college-age participants addresses two overarching questions: (a) how are immediate and delayed learning gains affected by the degree to which a lesson is embodied, and (b) how do the affordances of three different educational platforms affect immediate and delayed learning? Six 50 min-long lessons on centripetal force were created. The first factor was the degree of embodiment with two levels: (1) low and (2) high. The second factor was platform with three levels: (1) a large scale “mixed reality” immersive environment containing both digital and hands-on components called SMALLab, (2) an interactive whiteboard system, and (3) a mouse-driven desktop computer. Pre-tests, post-tests, and 1-week follow-up (retention or delayed learning gains) tests were administered resulting in a 2 × 3 × 3 design. Two knowledge subtests were analyzed, one that relied on more declarative knowledge and one that relied on more generative knowledge, e.g., hand-drawing vectors. Regardless of condition, participants made significant immediate learning gains from pre-test to post-test. There were no significant main effects or interactions due to platform or embodiment on immediate learning. However, from post-test to follow-up the level of embodiment interacted significantly with time, such that participants in the high embodiment conditions performed better on the subtest devoted to generative knowledge questions. We posit that better retention of certain types of knowledge can be seen over time when more embodiment is present during the encoding phase. This sort of retention may not appear on more traditional factual/declarative tests. Educational technology designers should consider using more sensorimotor feedback and gestural congruency when designing and opportunities for instructor professional development need to be provided as well.

SMALLab: Virtual Geology Studies Using Embodied Learning with Motion, Sound, and Graphics

We present a new and innovative interface that allows the learner’s body to move freely in a multimodal learning environment. The Situated Multimedia Arts Learning Laboratory (SMALLab) uses 3D object tracking, real time graphics, and surround‐sound to enhance embodied learning. Our hypothesis is that optimal learning and retention occur when learning is embodied, multiple modalities are recruited, and learners are allowed to be highly collaborative. We present two geology studies in a high school setting. In the layer cake scenario, students learned more after three days of building and discussing depositional layers in SMALLab than would be expected from a simple retest effect. In the contour mapping scenario, students learned significantly more after creating and discussing maps in the SMALLab environment for three days, compared with a group that received three days of regular instruction with hands‐on projects. We contend that embodied, collaborative, virtual environments have much to offer STEM learners, and the learning gains transcend those that can be expected from more traditional classroom procedures. SMALLab: Virtuelle Geologiestudien mit Hilfe der Ausdrack verliehenen Gelehrsamkeit mit Bewegong, Geräusch und graphischer Darstellung Wir zeigen eine neue und innovative Schnittstelle, die dem Körper des Anfängers erlaubt, sich frei in einer multimodalen Lernumgebung zu bewegen. Das situierte Multimedia Kunst‐Lern‐Labor (SMALLab) benutzt 3D‐Objektverfolgungen, reale zeitgraphische Darstellungen und Rundum‐Klangkulissen, um das erzielte Lernergebnis zu verbessern. Unsere Hypothese ist, dass optimale Lern‐ und Merkergebnisse erzielt werden, wenn “zu lernen” Ausdruck verliehen wird, mehrfache Verfahrensweisen ausgewählt sind, und Lernenden erlaubt wird, hoch kooperativ zu sein. Wir legen zwei Geologiestudien in einer High‐School Einstellung vor. Im “Schichtkuchen”‐Szenario lernten Studenten in drei Tagen mehr depositionale Schichten in SMALLab zu bauen und zu erörtern, als von einer einfachen Nachtestwirkung erwartet werden würde. Im Konturangleichungsszenario, lernten Studenten bedeutend mehr nach dem Entwerfen und Erörtern an drei Tagen von Landkarten in der SMALLab Umgebung, verglich mit einer Gruppe, die drei Tage regelmäßig Anweisungen bei der praktischen Projektarbeit erhielt. Wir behaupten, dass kooperative, virtuelle Umgebungen viel dazu beitragen, STEM‐Lernern Lerngewinne anzubieten, und dass sie jene übersteigen, die von mehr traditionellen Klassenzimmerverfahren erwartet werden können. SMALLab: étudier virtuellement la qéologie en passant par un apprentissage incorpe comportant le mouvement, le son et le graphisme Nous présentons un nouvel interface innovant qui permet au corps de l’apprenant de se déplacer librement dans un environnement d’apprentissage multimodal. Le Laboratoire d’Apprentissage des Arts Multimedia en Situation (SMALL) utilise le repérage des objets en 3D, du graphisme en temps réel, et une sonorisation enveloppante, ceci pour renforcer l’apprentissage « incorporé ». Notre hypothèse, c’est que l’apprentissage et la rétention sont les meilleurs lorsque l’apprentissage est « incorporé », lorsqu’on fait appel à des modalités multiples et lorsqu’on permet aux apprenants d’être des collaborateurs actifs. Nous présentons deux études de géologie dans un contexte de lycée.Dans le scénario du gâteau fourré, les élèves ont appris davantage après trois jours de construction et de débats sur les couches de sédimentation au SMALLab que ce qu’on pourrait attendre d’un simple effet de contre essai. Dans le scénario de cartographie des courbes de niveau les élèves ont appris nettement plus après avoir crée et débattu pendant trois jours sur les cartes dans l’environnement du SMALLab si on les compare à un groupe qui a reçu trois jours de formation traditionnelle avec des réalisations concrètes. Nous soutenons que les environnements virtuels, collaboratifs, « incorporés » offrent beaucoup de possibilités aux apprenants de Science et de Technologie et que les gains cognitifs dépassent ceux que l’on peut attendre des protocoles de classe plus traditionnels. SMALLab: como estudiar geología virtualmente a través de un apprendizaje “encarnado” incluyendo movimientos, sonido y grafismo Aquí presentamos un nuevo interface innovador que permite al cuerpo del alumno moverse libremente dentro de un entorno de aprendizaje multi‐modal. El Laboratorio de Aprendizaje de Artes Multimedia en Situación (SMALLab) utiliza la localización de objetos 3D, el grafismo en tiempo real y un sonido envolvente para realzar el aprendizaje «encarnado». Nuestra hipótesis es que los mejores niveles de aprendizaje y retención aparecen cuando el aprendizaje está «encarnado», cuando muchas modalidades están involucradas y cuando los discentes tienen une fuerte capacidad de colaboración. Aquí presentamos dos estudios de geología dentro de un contexto de enseñanza media. En el escenario del «pastel relleno» los alumnos han aprendido más al final de tres días de construcción y debate sobre las capas sedimentarias que lo que se puede esperar de un efecto de segunda prueba. En el escenario de cartografía de curvas de nivel, los alumnos aprendieron mucho más después de tres días de creación y de debate sobre mapas en el entorno del SMALLab en comparación con un grupo que había recibido tres días de aulas regulares con proyectos prácticos. Sostenemos que los entornos virtuales «encarnados» y colaborativos pueden ofrecer mucho a los alumnos de Ciencias y Tecnologías y que los beneficios cognitivos superan lo que se puede esperar de prácticas más tradiconales en el aula.

Embodied science and mixed reality: How gesture and motion capture affect physics education

A mixed design was created using text and game-like multimedia to instruct in the content of physics. The study assessed which variables predicted learning gains after a 1-h lesson on the electric field. The three manipulated variables were: (1) level of embodiment; (2) level of active generativity; and (3) presence of story narrative. Two types of tests were administered: (1) a traditional text-based physics test answered with a keyboard; and (2) a more embodied, transfer test using the Wacom large tablet where learners could use gestures (long swipes) to create vectors and answers. The 166 participants were randomly assigned to four conditions: (1) symbols and text; (2) low embodied; (3) high embodied/active; or (4) high embodied/active with narrative. The last two conditions were active because the on-screen content could be manipulated with gross body gestures gathered via the Kinect sensor. Results demonstrated that the three groups that included embodiment learned significantly more than the symbols and text group on the traditional keyboard post-test. When knowledge was assessed with the Wacom tablet format that facilitated gestures, the two active gesture-based groups scored significantly higher. In addition, engagement scores were significantly higher for the two active embodied groups. The Wacom results suggest test sensitivity issues; the more embodied test revealed greater gains in learning for the more embodied conditions. We recommend that as more embodied learning comes to the fore, more sensitive tests that incorporate gesture be used to accurately assess learning. The predicted differences in engagement and learning for the condition with the graphically rich story narrative were not supported. We hypothesize that a narrative effect for motivation and learning may be difficult to uncover in a lab experiment where participants are primarily motivated by course credit. Several design principles for mediated and embodied science education are proposed.

If the Gear Fits, Spin It!: Embodied Education and in-Game Assessments

Two embodied gears games were created. Better learners should use fewer gear switches to reflect their knowledge. Twenty-three 7th graders, playing as dyads, used gestures to manipulate virtual gears. The Kinect sensor tracked arm-spinning movements and switched gear diameters. Knowledge tests were administered. Statistically significant knowledge gains were seen. For Game 1 gear spun one direction, switching significantly predicted only pretest knowledge. For Game 2 gear spun two directions switching was also negatively correlated with both tests. For game 2, those who used fewer switches during gameplay understood the construct better scoring higher on both tests. Dyadic analyses revealed the winner used significantly fewer switches. In-process data can provide a window onto knowledge as it is being encoded. However, games should stay within the learners ZPD, because if the game is too easy Game 1, meaningful data may be difficult to gather. The use of in ludo data from games with high sensitivity may attenuate the need for repetitive traditional, post-intervention tests.

SMALLab: a mixed-reality environment for embodied and mediated learning

In this video presentation, we introduce the Situated Multimedia Arts Learning Lab [SMALLab], a mixed-reality learning environment that supports interactive engagement through full body 3D movements and gestures within a collaborative, computationally mediated space. The video begins by describing the holistic approach to embodied and mediated learning developed by our transdisciplinary research team, grounded in understandings derived from research in the learning sciences, digital media and human computer interaction. We then outline the three core tenets of effective learning exemplified by our research -- embodiment, multimodality and collaboration. The video next demonstrates the design and functionality of the physical and digital components of SMALLab. We conclude by illustrating our partner collaborations with K12 teachers and students with four scenarios depicting Geography, Physics, Language Arts and Chemistry learning modules.

Embodied Education in Mixed and Mediated Realties

This chapter provides a summary of some of this lab’s immersive media and embodied STEM learning research. It focuses on the integration of gesture in learning, and a new gesture-based assessment. A taxonomy for embodiment in education is included. The chapter concludes with several design principles that the Embodied Games Lab has culled over the years while creating educational content that maximizes the affordances of virtual and mixed reality technologies and meshes those with best pedagogical practices.

Immersive VR and Education: Embodied Design Principles that Include Gesture and Hand Controls

This article explores relevant applications of educational theory for the design of immersive virtual reality (VR). Two unique attributes associated with VR position the technology to positively affect education: (1) the sense of presence, and (2) the embodied affordances of gesture and manipulation in the 3rd dimension. These are referred to as the two profound affordances of VR. The primary focus of this article is on the embodiment afforded by gesture in 3D for learning. The new generation of hand controllers induces embodiment and agency via meaningful and congruent movements with the content to be learned. Several examples of gesture-rich lessons are presented. The final section includes an extensive set of design principles for immersive VR in education, and finishes with the Necessary Nine which are hypothesized to optimize the pedagogy within a lesson.