Seokmin Kang

Coordinating Gesture, Word, and Diagram: Explanations for Experts and Novices

Successful explanations are a symphony of gesture, language, and props. Here, we show how they are orchestrated in an experiment in which students explained complex systems to imagined novices and experts. Visual-spatial communication—diagram and gesture—was key; it represents thought more directly than language. The real or virtual diagrams created from gestures served as the stage for explanations, enriched by language and enlivened by deictic gestures to convey structure and iconic gestures to enact the behavior and functionality of the systems. Explanations to novices packed in more information than explanations to experts, emphasizing the information about action that is difficult for novices, and expressing information in multiple ways, using both virtual models created by gestures and visible ones.

Why Students Learn More From Dialogue- Than Monologue-Videos: Analyses of Peer Interactions

In 2 separate studies, we found that college-age students learned more when they collaboratively watched tutorial dialogue-videos than lecture-style monologue-videos. In fact, they can learn as well as the tutees in the dialogue-videos. These results replicate similar findings in the literature showing the advantage of dialogue-videos even when observers watched them individually. However, having the observing students watch collaboratively as dyads provided data to carry out in-depth analyses of their conversations and activities in order to understand why dialogue-videos are superior to monologue-videos. Toward that goal, transcripts of video dialogues and monologues, as well as peer-to-peer conversations of the observing students collected in a prior study, were analyzed using the ICAP (interactive, constructive, active, passive) framework as a lens. Three sets of analyses were carried out. The 1st set focused on the content of the videos in terms of the tutors’ and the tutees’ moves. The 2nd set focused on the activities and behaviors of the collaboratively observing dyads. The 3rd set focused on the role of the tutees in the dialogue-videos in eliciting constructive and interactive engagement from the observing students. Our analyses suggest that dialogue-videos naturally elicited more constructive and interactive engagement behaviors from the observers than the monologue-videos, which in turn mediated the observers’ own learning.

A Preliminary Evaluation of the Usability of an AI-Infused Orchestration System.

Artificial intelligence (AI) holds great promise for improving classroom orchestration—the teacher’s management of a classroom workflow that mixes small group, individual, and whole class activities. Although we have developed an orchestration system, named FACT, that uses AI, we were concerned that usability issues might decrease its effectiveness. We conducted an analysis of classroom video recordings that classified and compared the time FACT students spent to the time spent by students using paper versions of the same lessons. FACT wasted half the time that paper did. However FACT students spent slightly more time off task and had difficulties referring to objects on shared documents.

The Effect of Digital Versus Traditional Orchestration on Collaboration in Small Groups

We are developing an intelligent orchestration system named FACT (Formative Assessment using Computational Technology). Orchestration refers the teacher’s management of a face-to-face classroom workflow that mixes small group, individual and whole class activities. FACT is composed of an unintelligent Media system and an intelligent Analysis system. Although the Analysis system, which is still being refined, is designed to increase collaboration, prior work suggests that the Media system could possibly harm collaboration. Thus, we conducted an evaluation of the FACT Media system in classrooms, comparing it against traditional classrooms. We coded videos of small groups in order to measure their collaboration. The FACT Media system did no harm: the distribution of collaboration codes in FACT classrooms is statistically similar to the distribution in traditional classrooms. This null result is welcome news and sets the stage for testing the benefits of the Analysis system.