Seth Adjei

Improving Student Modeling Through Partial Credit and Problem Difficulty

Student modeling within intelligent tutoring systems is a task largely driven by binary models that predict student knowledge or next problem correctness (i.e., Knowledge Tracing (KT)). However, using a binary construct for student assessment often causes researchers to overlook the feedback innate to these platforms. The present study considers a novel method of tabling an algorithmically determined partial credit score and problem difficulty bin for each students current problem to predict both binary and partial next problem correctness. This study was conducted using log files from ASSISTments, an adaptive mathematics tutor, from the 2012-2013 school year. The dataset consisted of 338,297 problem logs linked to 15,253 unique student identification numbers. Findings suggest that an efficiently tabled model considering partial credit and problem difficulty performs about as well as KT on binary predictions of next problem correctness. This method provides the groundwork for modifying KT in an attempt to optimize student modeling.

Predicting student performance on post-requisite skills using prerequisite skill data: an alternative method for refining prerequisite skill structures

Prerequisite skill structures have been closely studied in past years leading to many data-intensive methods aimed at refining such structures. While many of these proposed methods have yielded success, defining and refining hierarchies of skill relationships are often difficult tasks. The relationship between skills in a graph could either be causal, therefore, a prerequisite relationship (skill A must be learned before skill B). The relationship may be non-causal, in which case the ordering of skills does not matter and may indicate that both skills are prerequisites of another skill. In this study, we propose a simple, effective method of determining the strength of pre-to-post-requisite skill relationships. We then compare our results with a teacher-level survey about the strength of the relationships of the observed skills and find that the survey results largely confirm our findings in the data-driven approach.

Improving Learning Maps Using an Adaptive Testing System: PLACEments

Several efforts have been put forth in finding algorithms for identifying optimal learning maps for a given cognitive domain. In (Adjei, et. al. 2014), we proposed a greedy search algorithm for searching data fitting models with equally accurate predictive power as the original skill graph, but with fewer nodes/skills in the graph. In this paper we present PLACEments, an adaptive testing system, and report on how it can be used to determine the strength of the prerequisite skill relationships in a given skill graph. We also present preliminary results that show that different learning maps need to be designed for students with different knowledge levels.

Sequencing content in an adaptive testing system: the role of choice

The effect of choice on student achievement and engagement has been an extensively researched area of learning analytics. Current research findings suggest a positive relationship between choice and varied outcome measures, but little has been reported to indicate whether these findings hold in the context of Intelligent Tutoring Systems (ITS). In this paper, we report the results of a randomized controlled experiment in which we investigate the effect of student choice on assignment completion and future achievement in an ITS. The experimental design uses three conditions to observe the effect of choice. In the first condition, students are able to choose the order in which to complete assignments, while in the second condition, students are prescribed an intuitive order in which to complete assignments. Those in the third condition were prescribed a counter-intuitive order in which to complete assignments. Results indicate that allowing students to choose the order in which to work on assignments leads to higher completion rates and better achievement at posttest. A post-hoc analysis also revealed that even considering students with similar completion rates, those given choice had higher posttest scores than those observed in any other condition. These results seem to support the many theories of the positive effect of choice on student achievement.

The Opportunity Count Model: A Flexible Approach to Modeling Student Performance

Detailed performance data can be exploited to achieve stronger student models when predicting next problem correctness (NPC) within intelligent tutoring systems. However, the availability and importance of these details may differ significantly when considering opportunity count (OC), or the compounded sequence of problems a student experiences within a skill. Inspired by this intuition, the present study introduces the Opportunity Count Model (OCM), a unique approach to student modeling in which separate models are built for differing OCs rather than creating a blanket model that encompasses all OCs. We use Random Forest (RF), which can be used to indicate feature importance, to construct the OCM by considering detailed performance data within tutor log files. Results suggest that OC is significant when modeling student performance and that detailed performance data varies across OCs.