Brian A. Jacob

The Impact of No Child Left Behind on Student Achievement.

The No Child Left Behind (NCLB) Act compelled states to design schoolaccountability systems based on annual student assessments. The effect of this Federal legislation on the distribution of student achievement is a highly controversial but centrally important question. This study presents evidence on whether NCLB has influenced student achievement based on an analysis of state-level panel data on student test scores from the National Assessment of Educational Progress (NAEP). The impact of NCLB is identified using a comparative interrupted time series analysis that relies on comparisons of the test-score changes across states that already had school-accountability policies in place prior to NCLB and those that did not. Our results indicate that NCLB generated statistically significant increases in the average math performance of 4 graders (effect size = 0.22 by 2007) as well as improvements at the lower and top percentiles. There is also evidence of improvements in 8 grade math achievement, particularly among traditionally low-achieving groups and at the lower percentiles. However, we find no evidence that NCLB increased reading achievement in either 4 or 8 grade. * We would like to thank Rob Garlick, Elias Walsh, Nathaniel Schwartz and Erica Johnson for their research assistance. We would also like to thank Kerwin Charles, Robert Kaestner, Ioana Marinescu and seminar participants at the Harris School of Public Policy and at the NCLB: Emerging Findings Research Conference for helpful comments. An earlier version of this work was also presented by Jacob as the David N. Kershaw Lecture at the Annual meeting of the Association of Public Policy and Management (November 2008). All errors are our own.

Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education

We examine how well principals can distinguish between more and less effective teachers. To put principal evaluations in context, we compare them with the traditional determinants of teacher compensation—education and experience—as well as value‐added measures of teacher effectiveness based on student achievement gains. We present “out‐of‐sample” predictions that mitigate concerns that the teacher quality and student achievement measures are determined simultaneously. We find that principals can generally identify teachers who produce the largest and smallest standardized achievement gains but have far less ability to distinguish between teachers in the middle of this distribution.

Are Idle Hands the Devil's Workshop? Incapacitation, Concentration and Juvenile Crime

This paper examines the short-term effect of school on juvenile crime. To do so, we bring together daily measures of criminal activity and detailed school calendar information from 29 jurisdictions across the country, and use the plausibly exogenous variation generated by teacher in-service days to estimate the school-crime relationship. We find that the level of property crime committed by juveniles decreases by 14 percent on days when school is in session, but that the level of violent crime increases by 28 percent on such days. These results do not appear to be driven by inflated reporting of crime on school days or substitution of crime across days. Our findings suggest that incapacitation and concentration influence juvenile crime - when juveniles are not engaged in supervised activities, they are more likely to engage in certain anti-social behaviors; at the same time, the increase in interactions associated with school attendance leads to more interpersonal conflict and violence. These results underscore the social nature of violent crime and suggest that youth programs - particularly those with no educational component such as midnight basketball or summer concerts - may entail important tradeoffs in terms of their effects on juvenile crime.

Public Housing, Housing Vouchers and Student Achievement: Evidence from Public Housing Demolitions in Chicago

There has been a substantial shift from public housing to voucher-based housing assistance over the past decade, largely in response to the rising cost of public housing and the high rates of crime, unemployment and school failure among public housing residents. Despite this shift, there is relatively little evidence on the impact of public housing or housing vouchers on educational outcomes. This paper utilizes a plausibly exogenous source of variation in housing assistance generated by public housing demolitions in Chicago to examine the impact of high-rise public housing on student outcomes. I find that children in households affected by the demolitions do no better or worse than their peers on a wide variety of achievement measures. Because the majority of households that leave public housing in response to the demolitions move to neighborhoods and schools that closely resemble those they left, the zero effect of the demolitions may be interpreted as the independent impact of public housing. These findings suggest that eliminating high-rise public housing will not necessarily lead to the benefits documented in housing mobility experiments such as Gautreaux or Moving to Opportunity

The Impact of Teacher Training on Student Achievement Quasi-Experimental Evidence from School Reform Efforts in Chicago

While there is a substantial literature on the relationship between general teacher characteristics and student learning, school districts and states often rely on in-service teacher training as a part of school reform efforts. Recent school reform efforts in Chicago provide an opportunity to examine in-service training using a quasi-experimental research design. In this paper, we use a regression discontinuity strategy to estimate the effect of teacher training on the math and reading performance of elementary students. We find that marginal increases in in-service training have no statistically or academically significant effect on either reading or math achievement, suggesting that modest investments in staff development may not be sufficient to increase the achievement of elementary school children in high-poverty schools.

Getting Tough? The Impact of High School Graduation Exams

The impact of high school graduation exams on student achievement and dropout rates is examined. Using data from the National Educational Longitudinal Survey (NELS), this analysis is able to control for prior student achievement and a variety of other student, school, and state characteristics. It was found that graduation tests have no significant impact on 12th-grade math or reading achievement. These results are robust with a variety of specification checks. Although graduation tests have no appreciable effect on the probability of dropping out for the average student, they increase the probability of dropping out among the lowest ability students. These results suggest that policymakers would be well advised to rethink current graduation test policies.

The Dynamics of Criminal Behavior: Evidence from Weather Shocks

While the persistence of criminal activity is well documented, this may be due to persistence in the unobserved determinants of crime. There are good reasons to believe, however, that there may actually be a negative relationship between crime rates in a particular area due to temporal displacement. We exploit the correlation between weather and crime to examine the short-run dynamics of crime. Using variation in lagged crime rates due to weather shocks, we find that the positive serial correlation is reversed. These findings suggest that the long-run impact of temporary crime-prevention efforts may be smaller than the short-run effects.

Principals as Agents: Subjective Performance Measurement in Education

In this paper, we compare subjective principal assessments of teachers to the traditional determinants of teacher compensation iV education and experience iV and another potential compensation mechanism -- value-added measures of teacher effectiveness based on student achievement gains. We find that subjective principal assessments of teachers predict future student achievement significantly better than teacher experience, education or actual compensation, though not as well as value-added teacher quality measures. In particular, principals appear quite good at identifying those teachers who produce the largest and smallest standardized achievement gains in their schools, but have far less ability to distinguish between teachers in the middle of this distribution and systematically discriminate against male and untenured faculty. Moreover, we find that a principalis overall rating of a teacher is a substantially better predictor of future parent requests for that teacher than either the teacheris experience, education and current compensation or the teacheris value-added achievement measure. These findings not only inform education policy, but also shed light on subjective performance assessment more generally.

The Impact of High-Stakes Testing in Chicago on Student Achievement in Promotional Gate Grades:

This article analyzes the impact of high-stakes testing in Chicago on student achievement in grades targeted for promotional decisions. Using a three-level Hierarchical Linear Model, we estimate achievement value added in gate grades (test-score increases over and above that predicted from a student’s prior growth trajectory) for successive cohorts of students and derive policy effects by comparing value added pre- and postpolicy. Test scores in these grades increased substantially following the introduction of high-stakes testing. The effects are larger in the 6th and 8th grades and smaller in the 3rd grade in reading. Effects are also larger in previously low-achieving schools. In reading, students with low skills experienced the largest improvement in learning gains in the year prior to testing, while students with skills closer to their grade level experienced the greatest benefits in mathematics.

The Persistence of Teacher-Induced Learning.

This paper constructs a statistical model of learning that suggests a systematic way of measuring the persistence of treatment effects in education. This method is straightforward to implement, allows for comparisons across educational treatments, and can be related to intuitive benchmarks. We demonstrate the methodology using student-teacher linked administrative data for North Carolina to examine the persistence of teacher quality. We find that teacher-induced learning has low persistence, with three-quarters or more fading out within one year. Other measures of teacher quality produce similar or lower persistence estimates.