Zeroing in on the best early-course metrics to identify at-risk students in general chemistry: an adaptive learning pre-assessment vs. traditional diagnostic exam

Abstract

ABSTRACT General chemistry are key gateway courses for many Science, Technology, Engineering, and Mathematics (STEM) majors. To improve student outcomes, it is critical both to identify at-risk students and implement evidence-based instructional strategies and interventions. These remain active areas of investigation, with recent literature showing that cognitive, affective and demographic characteristics all contribute as risk factors. Here we report an assessment of demographic characteristics and early-course cognitive measures in predicting student outcomes in first-term general chemistry (GC1), comparing an adaptive learning pre-assessment using ALEKS (Assessment and LEarning in Knowledge Spaces) with a traditional diagnostic exam. Student performance was evaluated across three sections (345 students) in a quasi-experimental pre-test/post-test design using a common ACS final exam. We find that ∼57% of variance in ACS final exam scores and ∼60% of variance in final course grades is related to three predictors: initial ALEKS knowledge check, diagnostic exam score, and predicted first year QPA, an internal metric based upon high school GPA, ACT/SAT scores, and college of admission. While no single metric could identify even one-half of students scoring lowest in the course (i.e. in the bottom 15%, consistent with the percentage of D’s, F’s and withdrawals in our sections), in combination ∼ 80% were identified.