Academic Outcomes of Students in University Digital Environment at Different Levels of Higher Education: Who Is More Successful?

Abstract

The problem of empirical assessment of various aspects of learning in the digital educational environment seems to be of particular relevance.At the same time, it is emphasized that there is a lack of comparative studies of students’ educational outcomes at different levels of higher education who have completed e-courses.The study was conducted at the Moscow State University of Psychology and Education (MSUPE), the sample size is N = 424 students.Subject of the study is immediate and long-term academic achievements of students in two levels of higher education who completed e-courses.Purpose of the study is to evaluate the differences in the academic achievements and knowledge retention of graduate and undergraduate students.Key findings of students’ academic outcomes comparative analysis are the following: (1) No differences were found between graduate and undergraduate students in the pretest, final test and overall e-course grade indicators.(2) The same tendency was revealed in students of both groups: pretest scores are low, posttest scores significantly and strongly increase, and then after 1.5—4 months they significantly decrease, while remaining significantly higher than the input indicators.The knowledge retention scores are very scattered in comparison with the direct ones.(3) The gain score effect size and the improvement index are significant for the final test only without adjusting for clustering, i.e.ignoring the fact that the sample consists of several student groups.A median graduate student would have a higher score than a median undergraduate student.Cluster-level effect size is not statistically significant.Cluster-level effect size for overall e-course grade indicators with difference-in-differences adjustment is also not reliable.(4) The knowledge retention scores in both students’ categories do not differ.The gain score effect sizes for knowledge retention, taking into account both the final test and the pretest, are not significant.(5) The psychometric characteristics of the academic achievement test in the field of empirical data quantitative analysis can be considered satisfactory.