Jesse R. Sparks

Simulated Dialogues with Virtual Agents: Effects of Agent Features in Conversation-Based Assessments.

Pedagogical agents are widely employed in intelligent tutoring systems and game-based learning environments, but research suggests that learning benefits from virtual agents vary as a function of agent features (e.g., the form or register of agent dialogue) and student characteristics (e.g., prior knowledge). Students’ responses to agent-based conversations provide useful evidence of students’ knowledge and skills for assessment purposes; however, it is unclear whether these agent design features and student characteristics similarly influence students’ interactions with agents in assessment (versus learning) contexts. In this paper, we explore relationships between agent features and student characteristics within a conversation-based assessment of science inquiry skills. We examined the effects of virtual agents’ knowledge status (low vs. high) and language use (comparative vs. argumentative question framing) on agent perceptions (ratings) in a conversation-based assessment of scientific reasoning (i.e., using data to predict the weather). Preliminary results show that the effects of these features on students’ perceptions of agents varied as a function of students’ background characteristics, consistent with research on learning from agents. Implications for designing agent-based assessments will be discussed.