Anthony F. Botelho

Improving Sensor-Free Affect Detection Using Deep Learning

Affect detection has become a prominent area in student modeling in the last decade and considerable progress has been made in developing effective models. Many of the most successful models have leveraged physical and physiological sensors to accomplish this. While successful, such systems are difficult to deploy at scale due to economic and political constraints, limiting the utility of their application. Examples of “sensor-free” affect detectors that assess students based solely using data on the interaction between students and computer-based learning platforms exist, but these detectors generally have not reached high enough levels of quality to justify their use in real-time interventions. However, the classification algorithms used in these previous sensor-free detectors have not taken full advantage of the newest methods emerging in the field. The use of deep learning algorithms, such as recurrent neural networks (RNNs), have been applied to a range of other domains including pattern recognition and natural language processing with success, but have only recently been attempted in educational contexts. In this work, we construct new “deep” sensor-free affect detectors and report significant improvements over previously reported models.

A Memory-Augmented Neural Model for Automated Grading

The need for automated grading tools for essay writing and open-ended assignments has received increasing attention due to the unprecedented scale of Massive Online Courses (MOOCs) and the fact that more and more students are relying on computers to complete and submit their school work. In this paper, we propose an efficient memory networks-powered automated grading model. The idea of our model stems from the philosophy that with enough graded samples for each score in the rubric, such samples can be used to grade future work that is found to be similar. For each possible score in the rubric, a student response graded with the same score is collected. These selected responses represent the grading criteria specified in the rubric and are stored in the memory component. Our model learns to predict a score for an ungraded response by computing the relevance between the ungraded response and each selected response in memory. The evaluation was conducted on the Kaggle Automated Student Assessment Prize (ASAP) dataset. The results show that our model achieves state-of-the-art performance in 7 out of 8 essay sets.

Predicting student performance on post-requisite skills using prerequisite skill data: an alternative method for refining prerequisite skill structures

Prerequisite skill structures have been closely studied in past years leading to many data-intensive methods aimed at refining such structures. While many of these proposed methods have yielded success, defining and refining hierarchies of skill relationships are often difficult tasks. The relationship between skills in a graph could either be causal, therefore, a prerequisite relationship (skill A must be learned before skill B). The relationship may be non-causal, in which case the ordering of skills does not matter and may indicate that both skills are prerequisites of another skill. In this study, we propose a simple, effective method of determining the strength of pre-to-post-requisite skill relationships. We then compare our results with a teacher-level survey about the strength of the relationships of the observed skills and find that the survey results largely confirm our findings in the data-driven approach.

Observing Personalizations in Learning: Identifying Heterogeneous Treatment Effects Using Causal Trees

The incorporation of computer-based platforms in the classroom has introduced the ability to conduct numerous randomized control trials at scale with student-level randomization. Such systems are able to collect vast amounts of data on each student while completing work in the classroom and at home. It is often the case, however, that the effects of these trials are reported across all students, ignoring the potential for personalized learning. Personalized learning, or the observation of heterogeneous treatment effects, considers that the effects of a studied learning intervention may differ for individual students; while an intervention may work well for low-performing students, for example, it may have no effect for higher performing students. Personalized learning can lead to better instructional practices that maximizes the learning benefits for each individual student, and with the use of computer-based platforms, such individualized instruction is made feasible at scale. In this work we use a causal decision tree to observe treatment effects in 9 experiments run in the ASSISTments online learning platform.

Incorporating Rich Features into Deep Knowledge Tracing

Knowledge Tracing aims to model student knowledge by predicting the correctness of each next item as students work through an assignment. Through recent developments in deep learning, Deep Knowledge Tracing (DKT) was explored as a method to improve upon traditional methods. Thus far, the DKT model has only considered the knowledge components and correctness as input, neglecting the other important features collected by computer-based learning platforms. This paper seeks to further improve upon DKT by incorporating more problem-level features. With this higher dimensional input, an adaption to the original DKT model structure is also proposed to convert the input into a low dimensional feature vector. Our results show that this adapted DKT model can effectively improve accuracy.

Sequencing content in an adaptive testing system: the role of choice

The effect of choice on student achievement and engagement has been an extensively researched area of learning analytics. Current research findings suggest a positive relationship between choice and varied outcome measures, but little has been reported to indicate whether these findings hold in the context of Intelligent Tutoring Systems (ITS). In this paper, we report the results of a randomized controlled experiment in which we investigate the effect of student choice on assignment completion and future achievement in an ITS. The experimental design uses three conditions to observe the effect of choice. In the first condition, students are able to choose the order in which to complete assignments, while in the second condition, students are prescribed an intuitive order in which to complete assignments. Those in the third condition were prescribed a counter-intuitive order in which to complete assignments. Results indicate that allowing students to choose the order in which to work on assignments leads to higher completion rates and better achievement at posttest. A post-hoc analysis also revealed that even considering students with similar completion rates, those given choice had higher posttest scores than those observed in any other condition. These results seem to support the many theories of the positive effect of choice on student achievement.