Thomas J. Kane

The Promise and Pitfalls of Using Imprecise School Accountability Measures

In recent years, most states have constructed elaborate accountability systems using school-level test scores. However, because the median elementary school contains only 69 children per grade level, such measures are quite imprecise. We evaluate the implications for school accountability systems. For instance, rewards or sanctions for schools with scores at either extreme primarily affect small schools and provide weak incentives to large ones. Nevertheless, we conclude that accountability systems may be worthwhile. Even in states with aggressive financial incentives, the marginal reward to schools for raising student performance is a small fraction of the potential labor market value for students.

Accountability and Flexibility in Public Schools: Evidence from Boston's Charters And Pilots

Charter schools are publicly funded but operate outside the regulatory framework and collective bargaining agreements characteristic of traditional public schools. In return for this freedom, charter schools are subject to heightened accountability. This paper estimates the impact of charter school attendance on student achievement using data from Boston, where charter schools enroll a growing share of students. We also evaluate an alternative to the charter model, Bostons pilot schools. These schools have some of the independence of charter schools, but operate within the school district, face little risk of closure, and are covered by many of same collective bargaining provisions as traditional public schools. Estimates using student assignment lotteries show large and significant test score gains for charter lottery winners in middle and high school. In contrast, lottery-based estimates for pilot schools are small and mostly insignificant. The large positive lottery-based estimates for charter schools are similar to estimates constructed using statistical controls in the same sample, but larger than those using statistical controls in a wider sample of schools. The latter are still substantial, however. The estimates for pilot schools are smaller and more variable than those for charters, with some significant negative effects.

Volatility in School Test Scores: Implications for Test-Based Accountability Systems

B y the spring of 2000, forty states had begun using student test scores to rate school performance. Twenty states have gone a step further and are attaching explicit monetary rewards or sanctions to a schools test performance. For example, California planned to spend

Teen Motherhood and Abortion Access

677 million on teacher incentives in 2001, providing bonuses of up to

Roe V. Wade and American Fertility

25,000 to teachers in schools with the largest test score gains. We highlight an under-appreciated weakness of school accountability systems—the volatility of test score measures—and explore the implications of that volatility for the design of school accountability systems. The imprecision of test score measures arises from two sources. The first is sampling variation, which is a particularly striking problem in elementary schools. With the average elementary school containing only sixty-eight students per grade level, the amount of variation stemming from the idiosyncrasies of the particular sample of students being tested is often large relative to the total amount of variation observed between schools. The second arises from one-time factors that are not sensitive to the size of the sample; for example, a dog barking in the playground on the day of the test, a severe flu season, a disruptive student in a class, or favorable chemistry between a group of students and their teacher. Both small samples and other one-time factors can add considerable volatility to test score measures.

School Accountability Ratings and Housing Values

We investigate the effect of abortion access on teen birthrates using county-level panel data. Past research suggested that prohibiting abortion led to higher teen birthrates. Perhaps surprisingly, we find that more recent restrictions in abortion access, including the closing of abortion clinics and restrictions on Medicaid funding, had the opposite effect. Small declines in access were related to small declines among in-wedlock births; out-of-wedlock births were relatively unaffected. Both results are consistent with a simple model in which pregnancy is endogenous and women gain new information about the attractiveness of parenthood only after becoming pregnant.

Gender and Performance: Evidence from School Assignment by Randomized Lottery

OBJECTIVES This article examines the effect of abortion legalization on fertility rates in the United States. METHODS Fertility rates were compared over time between states that varied in the timing of abortion legalization. RESULTS States legalizing abortion experienced a 4% decline in fertility relative to states where the legal status of abortion was unchanged. The relative reductions in births to teens, women more than 35 years of age, non-White women, and unmarried women were considerably larger. If women did not travel between states to obtain an abortion, the estimated impact of abortion legalization on birth rates would be about 11%. CONCLUSIONS A complete recriminalization of abortion nationwide could result in 440,000 additional births per year. A reversal of the Roe v Wade decision leaving abortion legal in some states would substantially limit this impact because of the extent of travel between states.

Does an Urban Teacher Residency Increase Student Achievement? Early Evidence From Boston:

During the past decade, states have constructed elaborate systems for rating the performance of individual schools based on student test scores and then have released this information in the form of school “report cards.” The federal No Child Left Behind Act of 2001 will accelerate that movement by requiring states to test all students in grades three through eight, publicly report each school’s student test performance, and sanction schools when they fail to achieve specific standards. Earlier research has documented the cross-sectional relationship between housing values and student test scores at neighborhood schools. 1 Given the magnitude of the relationship between test scores and housing prices in the cross section, one might expect the release of school report cards to have important effects on housing values and, indirectly, to provide incentives for schools to improve performance. However, there are also reasons to believe that the housing market would downplay the information in school report cards. In particular, school test scores are noisy measures of school performance and may provide homeowners with little new information about which schools are the best ones. 2

Higher Education Appropriations and Public Universities: Role of Medicaid and the Business Cycle

School choice programs are intended to improve student achievement, by allowing for better matches between students and schools. It is not clear, however, that academic achievement will improve if parents make school choice decisions over both academic and nonacademic school attributes (Justine S. Hastings et al., 2005, 2006). Indeed, many randomized studies of impacts of school choice find little or no effect of school choice on academic outcomes. For example, initial evaluations of randomized voucher experiments in Milwaukee and New York City found modest academic impacts on eligible students (John F. Witte et al., 1995; Daniel P. Mayer et al., 2002). More recently, evaluations of public school choice lotteries in Chicago and Charlotte have found no difference between the average lottery winner and loser in academic outcomes such as test scores (Julie Cullen et al., 2003; Hastings et al., 2006). When parents are choosing schools for academic and nonacademic reasons, school choice may increase utility but not necessarily improve academic outcomes. There is growing evidence that educational interventions may have heterogeneous treatment effects by gender. Analysis of the Moving To Opportunity demonstration, in which parents were randomly given the opportunity to move to nonpoverty neighborhoods, found improvements in education, mental health, and criminal behavior for females, but negative effects on males (Jeffrey R. Kling and Jeffrey B. Liebman, 2004). Similarly, Michael Anderson (2005) reanalyzed data from three randomized trials of early childhood education and found that all of the long-term benefits accrued to girls and not to boys. We use data from a public school choice program, with school assignment by lottery, to estimate the impacts on academic outcomes by race and gender of attending a first-choice school. Our data come from the CharlotteMecklenburg school district (CMS) in North Carolina, which introduced district-wide public school choice in the fall of 2002 after a racebased busing plan was terminated by the courts. The data include students’ choices, lottery numbers, school assignments, demographics, and academic achievement for the years surrounding implementation of school choice. We compare outcomes for those making similar choices, whose school assignment was determined solely by lottery number. Overall, there was no gain in academic achievement for those winning the lottery. White females did experience significant improvements in test scores when randomized into their first choice school, however. White females were also more likely to choose academically focused magnets and, among those who won the lottery, reported significant increases in time spent on homework. Our evidence suggests that school choice programs may have heterogeneous treatment effects by gender, which are related to differences in the factors driving parental choices.

Racial Test Score Differences as Evidence of Reverse Discrimination: Less than Meets the Eye

Boston Teacher Residency (BTR) is an innovative practice-based preparation program in which candidates work alongside a mentor teacher for a year before becoming a teacher of record in the Boston Public Schools (BPS). The authors found that BTR graduates are more racially diverse than other BPS novices, more likely to teach math and science, and more likely to remain teaching in the district through Year 5. Initially, BTR graduates for whom value-added performance data are available are no more effective at raising student test scores than other novice teachers in English language arts and less effective in math. The effectiveness of BTR graduates in math improves rapidly over time, however, such that by their 4th and 5th years they outperform veteran teachers. Simulations of the program’s overall effect through retention and effectiveness suggest that it is likely to improve student achievement in the district only modestly over the long run.