Bing-ru Teh

Do KIPP Schools Boost Student Achievement

The Knowledge Is Power Program (KIPP) is an influential and rapidly growing nationwide network of charter schools serving primarily disadvantaged minority students. Prominent elements of KIPPs educational model include high expectations for student achievement and behavior, and a substantial increase in time in school. KIPP is being watched closely by policy makers and educators as a possible model for urban education, but existing studies of KIPPs effects on students have been subject to methodological limitations, making them less than conclusive. We measure the achievement impacts of forty-one KIPP middle schools across the country, using propensity-score matching to identify traditional public school students with similar characteristics and prior-achievement histories as students who enter KIPP. We find consistently positive and statistically significant impacts of KIPP on student achievement, with larger impacts in math than reading. These impacts persist over four years following admission, and are not driven by attrition of low performers from KIPP schools.

Replicating Experimental Impact Estimates with Nonexperimental Methods in the Context of Control-Group Noncompliance

ABSTRACT A growing literature on within-study comparisons (WSC) examines whether and in what context nonexperimental methods can successfully replicate the results of randomized experiments. WSCs require that the experimental and nonexperimental methods assess the same causal estimand. But experiments that include noncompliance in treatment assignment produce a divergence in the causal estimands measured by standard approaches: the experiment-based estimate of the impact of treatment (the complier average causal effect, CACE) applies only to compliers, while the nonexperimental estimate applies to all subjects receiving treatment, including always-takers. We develop a new replication approach that solves this problem by using nonexperimental methods to produce an estimate that can be compared to the experimental intent-to-treat (ITT) impact estimate rather than the CACE. We demonstrate the applicability of the method in a WSC of the effects of charter schools on student achievement. In our example, some members of the randomized control group crossed over to treatment by enrolling in the charter schools. We show that several nonexperimental methods that incorporate pretreatment measures of the outcome of interest can successfully replicate experimental ITT impact estimates when control-group noncompliance (crossover) occurs—even when treatment effects differ for compliers and always takers.