Sébastien Lallé

Comparing Student Models in Different Formalisms by Predicting Their Impact on Help Success

We describe a method to evaluate how student models affect ITS decision quality – their raison d’etre. Given logs of randomized tutorial decisions and ensuing student performance, we train a classifier to predict tutor decision outcomes (success or failure) based on situation features, such as student and task. We define a decision policy that selects whichever tutor action the trained classifier predicts in the current situation is likeliest to lead to a successful outcome. The ideal but costly way to evaluate such a policy is to implement it in the tutor and collect new data, which may require months of tutor use by hundreds of students. Instead, we use historical data to simulate a policy by extrapolating its effects from the subset of randomized decisions that happened to follow the policy. We then compare policies based on alternative student models by their simulated impact on the success rate of tutorial decisions. We test the method on data logged by Project LISTEN’s Reading Tutor, which chooses randomly which type of help to give on a word. We report the cross-validated accuracy of predictions based on four types of student models, and compare the resulting policies’ expected success and coverage. The method provides a utility-relevant metric to compare student models expressed in different formalisms.

The Impact of Student Individual Differences and Visual Attention to Pedagogical Agents During Learning with MetaTutor

In this paper, we investigate the relationship between students’ (N = 28) individual differences and visual attention to pedagogical agents (PAs) during learning with MetaTutor, a hypermedia-based intelligent tutoring systems. We used eye tracking to capture visual attention to the PAs, and our results reveal specific visual attention-related metrics (e.g., fixation rate, longest fixations) that are significantly influenced by learning depending on student achievement goals. Specifically, performance-oriented students learned more with a long longest fixation and a high fixation rate on the PAs, whereas mastery-oriented students learned less with a high fixation rate on the PAs. Our findings contribute to understanding how to design PAs that can better adapt to student achievement goals and visual attention to the PA.

Further Results on Predicting Cognitive Abilities for Adaptive Visualizations

Previous work has shown that some user cognitive abilities relevant for processing information visualizations can be predicted from eye tracking data. Performing this type of user modeling is important for devising user-adaptive visualizations that can adapt to a user’s abilities as needed during the interaction. In this paper, we contribute to previous work by extending the type of visualizations considered and the set of cognitive abilities that can be predicted from gaze data, thus providing evidence on the generality of these findings. We also evaluate how quality of gaze data impacts prediction.

Impact of Individual Differences on Affective Reactions to Pedagogical Agents Scaffolding

Students’ emotions are known to influence learning and motivation while working with agent-based learning environments (ABLEs). However, there is limited understanding of how Pedagogical Agents (PAs) impact different students’ emotions, what those emotions are, and whether this is modulated by students’ individual differences (e.g., personality, goal orientation). Such understanding could be used to devise intelligent PAs that can recognize and adapt to students’ relevant individual differences in order to enhance their experience with learning environments. In this paper, we investigate the relationship between individual differences and students’ affective reactions to four intelligent PAs available in MetaTutor, a hypermedia-based intelligent tutoring system. We show that achievement goals and personality traits can significantly modulate students’ affective reactions to the PAs. These findings suggest that students may benefit from personalized PAs that could adapt to their motivational goals and personality.

Prediction of individual learning curves across information visualizations

Confident usage of information visualizations is thought to be influenced by cognitive aspects as well as amount of exposure and training. To support the development of individual competency in visualization processing, it is important to ascertain if we can track users’ progress or difficulties they might have while working with a given visualization. In this paper, we extend previous work on predicting in real time a user’s learning curve—a mathematical model that can represent a user’s skill acquisition ability—when working with a visualization. First, we investigate whether results we previously obtained in predicting users’ learning curves during visualization processing generalize to a different visualization. Second, we study to what extent we can make predictions on a user’s learning curve without information on the visualization being used. Our models leverage various data sources, including a user’s gaze behavior, pupil dilation, and cognitive abilities. We show that these models outperform a baseline that leverages knowledge on user task performance so far. Our best performing model achieves good accuracies in predicting users’ learning curves even after observing users’ performance on a few tasks only. These results represent an important step toward understanding how to support users in learning a new visualization.

Assistance in Building Student Models Using Knowledge Representation and Machine Learning

We propose a method and a first authoring tool to assist the design and implementation of diagnostic techniques. This method is independent from the domain and allows building more than one technique at once. The method is based on knowledge representation and a semi-automatic machine learning algorithm. We tested the method in two domains, surgery and reading English. Techniques built with our method beat the majority class in terms of accuracy.