Michael D. Wallis

Balancing Cognitive and Motivational Scaffolding in Tutorial Dialogue

A key challenge in the design of tutorial dialogue systems is identifying tutorial strategies that can effectively balance the tradeoffs between cognitive and affective student outcomes. This balance is problematic because the precise nature of the interdependence between cognitive and affective strategies is not well understood. Furthermore, previous studies suggest that some cognitive and motivational goals are at odds with one another because a tutorial strategy designed to maximize one may negatively impact the other. This paper reports on a tutorial dialogue study that investigates motivational strategies and cognitive feedback. It was found that the choice of corrective tutorial strategy makes a significant difference in the outcomes of both student learning gains and self-efficacy gains.

Characterizing the effectiveness of tutorial dialogue with hidden markov models

Identifying effective tutorial dialogue strategies is a key issue for intelligent tutoring systems research. Human-human tutoring offers a valuable model for identifying effective tutorial strategies, but extracting them is a challenge because of the richness of human dialogue. This paper addresses that challenge through a machine learning approach that 1) learns tutorial strategies from a corpus of human tutoring, and 2) identifies the statistical relationships between student outcomes and the learned strategies. We have applied hidden Markov modeling to a corpus of annotated task-oriented tutorial dialogue to learn one model for each of two effective human tutors. We have identified significant correlations between the automatically extracted tutoring modes and student learning outcomes. This work has direct applications in authoring data-driven tutorial dialogue system behavior and in investigating the effectiveness of human tutoring.

Modeling Dialogue Structure with Adjacency Pair Analysis and Hidden Markov Models

Automatically detecting dialogue structure within corpora of human-human dialogue is the subject of increasing attention. In the domain of tutorial dialogue, automatic discovery of dialogue structure is of particular interest because these structures inherently represent tutorial strategies or modes, the study of which is key to the design of intelligent tutoring systems that communicate with learners through natural language. We propose a methodology in which a corpus of human-human tutorial dialogue is first manually annotated with dialogue acts. Dependent adjacency pairs of these acts are then identified through X2 analysis, and hidden Markov modeling is applied to the observed sequences to induce a descriptive model of the dialogue structure.

Learner Characteristics and Feedback in Tutorial Dialogue

Tutorial dialogue has been the subject of increasing attention in recent years, and it has become evident that empirical studies of human-human tutorial dialogue can contribute important insights to the design of computational models of dialogue. This paper reports on a corpus study of human-human tutorial dialogue transpiring in the course of problem-solving in a learning environment for introductory computer science. Analyses suggest that the choice of corrective tutorial strategy makes a significant difference in the outcomes of both student learning gains and self-efficacy gains. The findings reveal that tutorial strategies intended to maximize student motivational outcomes (e.g., self-efficacy gain) may not be the same strategies that maximize cognitive outcomes (i.e., learning gain). In light of recent findings that learner characteristics influence the structure of tutorial dialogue, we explore the importance of understanding the interaction between learner characteristics and tutorial dialogue strategy choice when designing tutorial dialogue systems.

The impact of instructor initiative on student learning: a tutoring study

In the quest to find instructional approaches that benefit student learning, engagement, and retention, evidence suggests providing students with hands-on practice is a worthwhile use of class time. This paper presents results from an exploratory study of two different instructional approaches that were encountered in a study of experienced human tutors working with novice computing students engaged in a programming exercise. No difference in average learning gains was found between a moderate approach, in which students were given control of problem solving nearly half the time, and a proactive approach in which the tutor took initiative nearly three-fourths of the time. Implications of this finding for fine-grained instructional strategy, as well as for broader classroom management decisions, are discussed. This paper also makes the case for the value of one-on-one tutoring studies as an exploratory research methodology for the comparative evaluation of computer science teaching strategies.

Investigating the role of student motivation in computer science education through one-on-one tutoring

The majority of computer science education research to date has focused on purely cognitive student outcomes. Understanding the motivational states experienced by students may enhance our understanding of the computer science learning process, and may reveal important instructional interventions that could benefit student engagement and retention. This article investigates issues of student motivation as they arise during one-on-one human tutoring in introductory computer science. The findings suggest that the choices made during instructional discourse are associated with cognitive and motivational outcomes, and that particular strategies can be leveraged based on an understanding of the student motivational state.

Inferring Tutorial Dialogue Structure with Hidden Markov Modeling

The field of intelligent tutoring systems has seen many successes in recent years. A significant remaining challenge is the automatic creation of corpus-based tutorial dialogue management models. This paper reports on early work toward this goal. We identify tutorial dialogue modes in an unsupervised fashion using hidden Markov models (HMMs) trained on input sequences of manually-labeled dialogue acts and adjacency pairs. The two best-fit HMMs are presented and compared with respect to the dialogue structure they suggest; we also discuss potential uses of the methodology for future work.