Brent J. Evans

Persistence Patterns in Massive Open Online Courses (MOOCs)

Using a unique dataset of 44 Massive Open Online Courses (MOOCs), this article examines critical patterns of enrollment, engagement, persistence, and completion among students in online higher education. By leveraging fixed-effects specifications based on over 2.1 million student observations across more than 2,900 lectures, we analyzed engagement, persistence, and completion rates at the student, lecture, and course levels. We found compelling and consistent temporal patterns: across all courses, participation declines rapidly in the first week but subsequently flattens out in later weeks of the course. However, this decay is not entirely uniform. We also found that several student and lecturespecific traits were associated with student persistence and engagement. For example, the sequencing of a lecture within a batch of released videos as well as its title wording were related to student watching. We also saw consistent patterns in how student characteristics are associated with persistence and completion. Students were more likely to complete the course if they completed a pre-course survey or followed a quantitative track (as opposed to qualitative or auditing track) when available. These findings suggest potential course design changes that are likely to increase engagement, persistence, and completion in this important, new educational setting.

The Limitations of the GRE in Predicting Success in Biomedical Graduate School

Historically, admissions committees for biomedical Ph.D. programs have heavily weighed GRE scores when considering applications for admission. The predictive validity of GRE scores on graduate student success is unclear, and there have been no recent investigations specifically on the relationship between general GRE scores and graduate student success in biomedical research. Data from Vanderbilt University Medical School’s biomedical umbrella program were used to test to what extent GRE scores can predict outcomes in graduate school training when controlling for other admissions information. Overall, the GRE did not prove useful in predicating who will graduate with a Ph.D., pass the qualifying exam, have a shorter time to defense, deliver more conference presentations, publish more first author papers, or obtain an individual grant or fellowship. GRE scores were found to be moderate predictors of first semester grades, and weak to moderate predictors of graduate GPA and some elements of a faculty evaluation. These findings suggest admissions committees of biomedical doctoral programs should consider minimizing their reliance on GRE scores to predict the important measures of progress in the program and student productivity.

Behavior Prediction in MOOCs using Higher Granularity Temporal Information

In this paper, we present early research evaluating the predictive power of a variety of temporal features across student subpopulations with distinctive behaviors at the beginning of the course. Initial results illustrate that these features predict important differences across the subpopulations and over time in the courses. Ultimately, these results have implications for effectively targeting adaptive scaffolding tailored to the particular intentions and goals of subpopulations in MOOCs.

Understanding Loan Aversion in Education: Evidence from High School Seniors, Community College Students, and Adults

Although prior research has suggested that some students may be averse to taking out loans to finance their college education, there is little empirical evidence showing the extent to which loan aversion exists or how it affects different populations of students. This study provides the first large-scale quantitative evidence of levels of loan aversion in the United States. Using survey data collected on more than 6,000 individuals, we examine the frequency of loan aversion in three distinct populations. Depending on the measure, between 20 and 40% of high school seniors exhibit loan aversion with lower rates among community college students and adults not in college. Women are less likely to express loan-averse attitudes than men, and Hispanic respondents are more likely to be loan averse than White respondents.

A Randomized Experiment Testing the Efficacy of a Scheduling Nudge in a Massive Open Online Course (MOOC)

An increasing number of students are taking classes offered online through open-access platforms; however, the vast majority of students who start these classes do not finish. The incongruence of student intentions and subsequent engagement suggests that self-control is a major contributor to this stark lack of persistence. This study presents the results of a large-scale field experiment (N = 18,043) that examines the effects of a self-directed scheduling nudge designed to promote student persistence in a massive open online course. We find that random assignment to treatment had no effects on near-term engagement and weakly significant negative effects on longer-term course engagement, persistence, and performance. Interestingly, these negative effects are highly concentrated in two groups of students: those who registered close to the first day of class and those with .edu e-mail addresses. We consider several explanations for these findings and conclude that theoretically motivated interventions may interact with the diverse motivations of individual students in possibly unintended ways.

Lessons learned from a data-driven college access program: The National College Advising Corps.

This chapter discusses the collaboration between a national college access program, the National College Advising Corps (NCAC), and its research and evaluation team at Stanford University. NCAC is currently active in almost four hundred high schools and through the placement of a recent college graduate to serve as a college adviser provides necessary information and support for students who may find it difficult to navigate the complex college admission process. The advisers also conduct outreach to underclassmen in an effort to improve the school-wide college-going culture. Analyses include examination of both quantitative and qualitative data from numerous sources and partners with every level of the organization from the national office to individual high schools. The authors discuss balancing the pursuit of evaluation goals with academic scholarship. In an effort to benefit other programs seeking to form successful data-driven interventions, the authors provide explicit examples of the partnership and present several examples of how the program has benefited from the data gathered by the evaluation team.

How Financial Literacy, Federal Aid Knowledge, and Credit Market Experience Predict Loan Aversion for Education:

Many students are averse to taking out loans to pay for education—a phenomenon that is commonly discussed but rarely systematically analyzed. This study explores the relationship between student loan aversion and individual financial characteristics. In this analysis, we rely on a unique dataset of survey responses from more than 5,000 high school seniors, community college students who did and did not borrow for higher education, and adults without a college degree. Regression analyses, using a robust set of controls and institutional fixed effects, show that higher financial literacy and higher knowledge of federal student loans are related to lower loan aversion for education. The magnitude of these effects is large, as much as a 30 to 50 percent reduction in loan aversion in some samples. There is also evidence that prior experience with payday lending is related to increased loan aversion for community college students who did not borrow for college.