Juan Miguel L. Andres

Studying MOOC completion at scale using the MOOC replication framework

Research on learner behaviors and course completion within Massive Open Online Courses (MOOCs) has been mostly confined to single courses, making the findings difficult to generalize across different data sets and to assess which contexts and types of courses these findings apply to. This paper reports on the development of the MOOC Replication Framework (MORF), a framework that facilitates the replication of previously published findings across multiple data sets and the seamless integration of new findings as new research is conducted or new hypotheses are generated. In the proof of concept presented here, we use MORF to attempt to replicate 15 previously published findings across 29 iterations of 17 MOOCs. The findings indicate that 12 of the 15 findings replicated significantly across the data sets, and that two findings replicated significantly in the opposite direction. MORF enables larger-scale analysis of MOOC research questions than previously feasible, and enables researchers around the world to conduct analyses on huge multi-MOOC data sets without having to negotiate access to data.

Affect dynamics in military trainees using vMedic: From engaged concentration to boredom to confusion

The role of affect in learning has received increasing attention from AIED researchers seeking to understand how emotion and cognition interact in learning contexts. The dynamics of affect over time have been explored in a variety of research environments, allowing researchers to determine the extent to which common patterns are captured by hypothesized models. This paper present an analysis of affect dynamics among learners using vMedic, which teaches combat medicine protocols as part of the military training at West Point, the United States Military Academy. In doing so, we seek both to broaden the variety of learning contexts being explored in order better understand differences in these patterns and to test the theoretical predictions on the development of affect over time.

Replicating MOOC predictive models at scale

We present a case study in predictive model replication for student dropout in Massive Open Online Courses (MOOCs) using a large and diverse dataset (133 sessions of 28 unique courses offered by two institutions). This experiment was run on the MOOC Replication Framework (MORF), which makes it feasible to fully replicate complex machine learned models, from raw data to model evaluation. We provide an overview of the MORF platform architecture and functionality, and demonstrate its use through a case study. In this replication of [41], we contextualize and evaluate the results of the previous work using statistical tests and a more effective model evaluation scheme. We find that only some of the original findings replicate across this larger and more diverse sample of MOOCs, with others replicating significantly in the opposite direction. Our analysis also reveals results which are highly relevant to the prediction task which were not reported in the original experiment. This work demonstrates the importance of replication of predictive modeling research in MOOCs using large and diverse datasets, illuminates the challenges of doing so, and describes our freely available, open-source software framework to overcome barriers to replication.

Wheel-Spinning in a Game-Based Learning Environment for Physics

We study wheel-spinning behavior among students using an educational game for physics. We attempted to determine whether students wheel-spin, and to build a wheel-spinning detector. We found that about 30 to 40i?ź% of students are unable to successfully complete a level when attempting it 8 times or more, or when working on it for more than 160i?źs. We also found that past performance is predictive of wheel-spinning, and that persistence increases both the likelihood of success and of wheel-spinning. Finally, we found that wheel-spinning in this context is different from wheel-spinning exhibited in prior work in that it is relatively easy to detect and does not suffer from cold starts.