Matthew Jensen Hays

The costs and benefits of providing feedback during learning.

It seems uncontroversial that providing feedback after a test, in the form of the correct answer, enhances learning. In real-world educational situations, however, the time available for learning is often constrained— and feedback takes time. We report an experiment in which total time for learning was fixed, thereby creating a trade-off between spending time receiving feedback and spending time on other learning activities. Our results suggest that providing feedback is not universally beneficial. Indeed, under some circumstances, taking time to provide feedback can have a negative net effect on learning. We also found that learners appear to have some insight about the costs of feedback; when they were allowed to control feedback, they often skipped unnecessary feedback in favor of additional retrieval attempts, and they benefited from doing so. These results underscore the importance of considering the costs and benefits of interventions designed to enhance learning.

Feedback Specificity and the Learning of Intercultural Communication Skills

The role of explicit feedback in learning has been studied from a variety of perspectives and in many contexts. In this paper, we examine the impact of the specificity of feedback delivered by an intelligent tutoring system in a game-based environment for cultural learning. We compared two versions: one that provided only “bottom-out” hints and feedback versus one that provided only conceptual messages. We measured during-training performance, in-game transfer, and long-term retention. Consistent with our hypotheses, specific feedback utterances produced inferior learning on the in-game transfer task when compared to conceptual utterances. No differences were found on a web-based post-test. We discuss possible explanations for these findings, particularly as they relate to the learning of loosely defined skills and serious games.

Investigating the relationship between presence and learning in a serious game

We investigate the role of presence in a serious game for intercultural communication and negotiation skills by comparing two interfaces: a 3D version with animated virtual humans and sound against a 2D version using text-only interactions with static images and no sound. Both versions provide identical communicative action choices and are driven by the same underlying simulation engine. In a study, the 3D interface led to a significantly greater self-reported sense of presence, but produced significant, but equivalent learning on immediate posttests for declarative and conceptual knowledge related to intercultural communication. Log data reveals that 3D learners needed fewer interactions with the system than those in the 2D environment, suggesting they benefited equally with less practice and may have treated the experience as more authentic.

Situated Pedagogical Authoring: Authoring Intelligent Tutors from a Student’s Perspective

We describe the Situated Pedagogical Authoring (SitPed) system that seeks to allow non-technical authors to create ITS content for soft-skills training, such as counseling skills. SitPed is built on the assertion that authoring tools should use the learner’s perspective to the greatest extent possible. SitPed provides tools for creating tasks lists, authoring assessment knowledge, and creating tutor messages. We present preliminary findings of a two-phase study comparing authoring in SitPed to an ablated version of the same system and a spreadsheet-based control. Findings suggest modest advantages for SitPed in terms of the quality of the authored content and student learning.