Lindsay C. Page

Freshman Year Financial Aid Nudges: An Experiment to Increase FAFSA Renewal and College Persistence

In this paper we investigate, through a randomized controlled trial design, the impact of a personalized text messaging intervention designed to encourage college freshmen to refile their Free Application for Federal Student Aid (FAFSA) and maintain their financial aid for sophomore year. The intervention produced large and positive effects among freshmen at community colleges where text recipients were almost 14 percentage points more likely to remain continuously enrolled through the Spring of sophomore year. By contrast, the intervention did not improve sophomore year persistence among freshmen at four-year institutions among whom the rate of persistence was already high.

Compared to what? Variation in the impacts of early childhood education by alternative care type

Early childhood education research often compares a group of children who receive the intervention of interest to a group of children who receive care in a range of different care settings. In this paper, we estimate differential impacts of an early childhood intervention by alternative care setting, using data from the Head Start Impact Study, a large-scale randomized evaluation. To do so, we utilize a Bayesian principal stratification framework to estimate separate impacts for two types of Compliers: those children who would otherwise be in other center-based care when assigned to control and those who would otherwise be in home-based care. We find strong, positive short-term effects of Head Start on receptive vocabulary for those Compliers who would otherwise be in home-based care. By contrast, we find no meaningful impact of Head Start on vocabulary for those Compliers who would otherwise be in other center-based care. Our findings suggest that alternative care type is a potentially important source of variation in early childhood education interventions.

Principal Stratification as a Framework for Investigating Mediational Processes in Experimental Settings

Abstract Experimental evaluations are increasingly common in the U.S. educational policy-research context. Often, in investigations of multifaceted interventions, researchers and policymakers alike are interested in not only whether a given intervention impacted an outcome but also why. What features of the intervention led to the impacts observed, or what was the causal mechanism or pathway through which treatment assignment resulted in an improved outcome? Quantitative methods for modeling such mediational processes appropriately are an active area of methodological exploration. I contribute to this literature by discussing an approach that relies on the analytic framework of principal stratification. Approaches, such as regression analysis and instrumental variables estimation (a special case of principal stratification), rely on assumptions—sequential ignorability and an exclusion restriction, respectively—that may be too strong to be plausible in practical settings. Principal stratification provides a broader framework with which these assumptions may be relaxed, although in exchange for other sets of assumptions. To illustrate and critique, I apply the principal stratification framework and Bayesian estimation techniques to data from MDRCs experimental evaluation of career academy high schools to explore the hypothesis that students’ increased exposure to the world-of-work through the career academy treatment serves as a causal mechanism of subsequent improvements in earnings in the years following high school. Whereas many applications of the principal stratification framework consider a mediator measured on a binary scale, this article contributes an example in which a three-category measure of the mediator is considered.

Principal Stratification A Tool for Understanding Variation in Program Effects Across Endogenous Subgroups

Increasingly, researchers are interested in questions regarding treatment-effect variation across partially or fully latent subgroups defined not by pretreatment characteristics but by postrandomization actions. One promising approach to address such questions is principal stratification. Under this framework, a researcher defines endogenous subgroups, or principal strata, based on post-randomization behaviors under both the observed and the counterfactual experimental conditions. These principal strata give structure to such research questions and provide a framework for determining estimation strategies to obtain desired effect estimates. This article provides a nontechnical primer to principal stratification. We review selected applications to highlight the breadth of substantive questions and methodological issues that this method can inform. We then discuss its relationship to instrumental variables analysis to address binary noncompliance in an experimental context and highlight how the framework can be generalized to handle more complex posttreatment patterns. We emphasize the counterfactual logic fundamental to principal stratification and the key assumptions that render analytic challenges more tractable. We briefly discuss technical aspects of estimation procedures, providing a short guide for interested readers.

Middle School Math Acceleration and Equitable Access to 8th Grade Algebra: Evidence from the Wake County Public School System

Taking algebra by eighth grade is considered an important milestone on the pathway to college readiness. We highlight a collaboration to investigate one district’s effort to increase middle school algebra course-taking. In 2010, the Wake County Public Schools began assigning middle school students to accelerated math and eighth-grade algebra based on a defined prior achievement metric. This policy reduced the relationship between course assignment and student characteristics such as income and race/ethnicity, while increasing its relationship to academic skill. The policy increased the share of students on track for algebra by eighth grade. Students placed in accelerated math were exposed to higher-skilled peers but larger classes. Future work will assess impacts on subsequent achievement and course-taking outcomes.School districts across the country have struggled to increase the proportion of students taking algebra by 8th grade, thought to be an important milestone on the pathway to college preparedness. We highlight key features of a research collaboration between the Wake County Public School System and Harvard University that have enabled investigation of one such effort to solve this problem. In 2010, the district began assigning middle school students to accelerated math coursework leading to 8th grade algebra on the basis of a clearly defined measured of prior academic skill. We document two important facts. First, use of this new rule greatly reduced the relationship between course assignment and student factors such as income and race while increasing the relationship between course assignment and academic skill. Second, using a regression discontinuity analytic strategy, we show that the assignment rule had strong impacts on the fraction of students on track to complete algebra by 8th grade. Students placed in accelerated math were exposed to higher-skilled peers but larger class sizes. We describe future plans for assessing impacts on achievement and high school course-taking outcomes.

Does School Policy Affect Housing Choices? Evidence From the End of Desegregation in Charlotte–Mecklenburg:

We examine whether the legal decision to grant unitary status to the Charlotte–Mecklenburg school district, which led to the end of race-conscious student assignment policies, increased the probability that families with children enrolled in the district would move to neighborhoods with a greater proportion of student residents of the same race as their own children. Motivated by the rich but inconclusive literature on the consequences of educational and residential segregation, we make use of a natural policy experiment—a judicial decision to end court-ordered busing—to estimate the causal impacts of this policy shift on household residential decisions. We find that, for those who moved, the legal decision made White families with children in the Charlotte–Mecklenburg Schools substantially more likely than they were during desegregation to move to a neighborhood with a greater proportion of White residents than their own neighborhood.

Surprising Ripple Effects: How Changing the SAT Score-Sending Policy for Low-Income Students Impacts College Access and Success

A growing economics literature reveals that small and subtle policy adjustments can induce relatively large “ripple effects.” We contribute to this literature by evaluating a College Board initiative, launched in the fall of 2007, which increased the number of free official SAT score reports afforded to low-income students and changed the time horizon over which these free score sends could be used. By resetting the default number of free SAT score reports from four to eight for SAT fee-waiver recipients, the College Board hoped to increase the number of college applications submitted by these students and to improve their college match. Using a difference-in-differences analytic strategy, we show that low-income students took advantage of this policy and were roughly 10 percentage points more likely to send eight or more score reports. We find that this policy achieved its intended goal of increasing college access and that it also favorably impacted college completion rates. Specifically, we estimate that inducing a low-income student to send one more score report, on average, increased on-time college attendance by nearly 5 percentage points and five-year bachelor’s completion by slightly more than 3 percentage points. The policy impact was driven entirely by students who, based on SAT scores, were competitive candidates for admission to four-year colleges.

Surprising Ripple Effects

Subtle policy adjustments can induce relatively large “ripple effects.” We evaluate a College Board initiative that increased the number of free SAT score reports available to low-income students and changed the time horizon for using these score reports. Using a difference-in-differences analytic strategy, we estimate that targeted students were roughly 10 percentage points more likely to send eight or more reports. The policy improved on-time college attendance and 6-year bachelor’s completion by about 2 percentage points. Impacts were realized primarily by students who were competitive candidates for 4-year college admission. The bachelor’s completion impacts are larger than would be expected based on the number of students driven by the policy change to enroll in college and to shift into more selective colleges. The unexplained portion of the completion effects may result from improvements in nonacademic fit between students and the postsecondary institutions in which they enroll.

Customized Nudging to Improve FAFSA Completion and Income Verification.

For most students from low- or moderate-income families, successfully completing the Free Application for Federal Student Aid (FAFSA) is a crucial gateway on the path to college access. Yet, researchers have long recognized that the complexity of the FAFSA can serve as a barrier to students applying for — and in turn receiving — financial aid. We investigate the impact of a texting campaign specifically to address the informational and behavioral barriers associated with initial FAFSA filing. We advance beyond prior work by designing a texting system that leveraged regularly-updated administrative data on the status of students’ FAFSA submissions to provide students with personalized outreach and updates on their FAFSA completion status. Students were able to write back for one-on-one, text-based assistance. We implemented the intervention in two distinct locations. In partnership with a set of eight school districts in Texas that together serve over 17,000 high school seniors, we tested the impact of this intervention through a school-level randomized trial. In partnership with the state of Delaware, we offered the text-based outreach to all high school seniors in Delaware public high schools, and we tested the impact of this statewide effort using a quasi-experimental matching strategy. Evidence from both sites indicates that the text-based outreach serves to improve FAFSA filing outcomes. In Delaware, rates of FAFSA completion were improved overall. In Texas, students filed the FAFSA earlier as a result of the outreach. In Texas, where we are able to observe subsequent college enrollment, the outreach impacted immediate college matriculation by four percentage points. We consider two potential mechanisms beyond simply completing the FAFSA through which the intervention could have improved rates of college enrollment. First, as a result of earlier filing, students may have accessed more generous financial aid. Second, the messaging may have made more salient the income verification process for the sizeable share of filers selected for verification. Earlier filing also may have afforded these students more time to navigate the verification process. We provide evidence to support both of these potential channels.

Understanding the impact of career academy attendance: an application of the principal stratification framework for causal effects accounting for partial compliance.

Background: Results from MDRCs longitudinal, random-assignment evaluation of career-academy high schools reveal that several years after high-school completion, those randomized to receive the academy opportunity realized a