Douglas N. Harris

Accountability, Standards, and the Growing Achievement Gap: Lessons from the Past Half‐Century

The rise of accountability policies during the early 1990s coincided with an increase in the achievement gap between white and minority students, reversing decades of steady improvement in outcome equity. This article explores the policies that helped to reduce the achievement gap before 1990, the effects of the subsequent shift toward accountability, and what can be learned from past successes to guide the future development of accountability systems. An extensive review of research suggests that pre‐1990s reductions in the achievement gap occurred because minority students were exposed to greater resources and academic content. We find little evidence that most forms of accountability have placed any downward pressure on the achievement gap, suggesting that the upward trend in the gap during the 1990s may be more than a coincidence. The few forms of accountability that have apparently helped to improve equity, especially promotion‐graduation exams for students, have in common with past successful policies the effect of increasing student exposure to resources and/or content. This suggests that accountability can help to improve educational equity but that such a change must be based on some basic assumptions that are inconsistent with much of the current reform movement. Specifically, A Nation at Risk has important lessons for No Child Left Behind and state‐level accountability programs.

Mix and Match: What Principals Really Look for When Hiring Teachers

The vast majority of research and policy related to teacher quality focuses on the supply of teachers and ignores teacher demand. In particular, the important role of school principals in hiring teachers is rarely considered. Using interviews of school principals in a midsized Florida school district, we provide an exploratory mixed methods analysis of the teacher characteristics principals prefer. Our findings contradict the conventional wisdom that principals undervalue content knowledge and intelligence. Principals in our study ranked content knowledge third among a list of twelve characteristics. Intelligence does appear less important at first glance, but this is apparently because principals believe all applicants who meet certification requirements meet a minimum threshold on intelligence and because some intelligent teachers have difficulty connecting with students. More generally, we find that principals prefer an individual mix of personal and professional qualities. They also create an organizational mix, hiring teachers who differ from those already in the school in terms of race, gender, experience, and skills, and an organizational match, in which teachers have similar work habits and a high propensity to remain with the school over time. Because of tenure rules, many principals also prefer less experienced (untenured) teachers, even though research suggests that they are less effective.

Would Accountability Based on Teacher Value Added Be Smart Policy? An Examination of the Statistical Properties and Policy Alternatives

Annual student testing may make it possible to measure the contributions to student achievement made by individual teachers. But would these teacher value-added measures help to improve student achievement? I consider the statistical validity, purposes, and costs of teacher value-added policies. Many of the key assumptions of teacher value added are rejected by empirical evidence. However, the assumption violations may not be severe, and value-added measures still seem to contain useful information. I also compare teacher value-added accountability with three main policy alternatives: teacher credentials, school value-added accountability, and formative uses of test data. I argue that using teacher value-added measures is likely to increase student achievement more efficiently than a teacher credentials-only strategy but may not be the most cost-effective policy overall. Resolving this issue will require a new research and policy agenda that goes beyond analysis of assumptions and statistical properties and focuses on the effects of actual policy alternatives.

How Principals "Bridge and Buffer" the New Demands of Teacher Quality and Accountability: A Mixed-Methods Analysis of Teacher Hiring

In this mixed‐methods study, we examine the degree to which district‐ and building‐level administrators accommodate teacher‐quality and test‐based accountability policies in their hiring practices. We find that administrators negotiated local hiring goals with characteristics emphasized by federal and state teacher‐quality policies, such as knowledge of the subject and teaching skills. While district administrators and principals largely “bridged” to external certification requirements, some principals “buffered” their hiring decisions from the pressures of test‐based accountability. Principals who bridged to test‐based accountability gave greater weight to subject knowledge and teaching skills. We find that bridging and buffering differs by policy and cannot be easily applied to accountability policies. Specifically, separating the indirect effect of external accountability from other policies influencing principal hiring is difficult. Our analysis also highlights tensions among local, state, and federal policies regarding teacher quality and the potential of accountability to permeate noninstructional school decision making.

High‐Flying Schools, Student Disadvantage, and the Logic of NCLB

In debates about accountability, advocates often point to individual “high‐flying” schools that achieve high test scores despite serving disadvantaged populations. Using a near‐census of U.S. public schools, this new analysis considers the likelihood that schools become high flyers. The results suggest that of the more than 60,000 schools considered, low‐poverty schools are 22 times more likely to reach consistently high academic achievement compared with high‐poverty schools. Schools serving student populations that are both low poverty and low minority are 89 times more likely to be consistently high performing compared with high‐poverty, high‐minority schools. This does not mean that schools have no influence over student achievement, or that schools should be unaccountable, but that accountability systems need to have carefully defined objectives and measures of school progress. In addition, the results suggest the continued need to address home and community factors in the pursuit of educational equity.

How Teacher Evaluation Methods Matter for Accountability A Comparative Analysis of Teacher Effectiveness Ratings by Principals and Teacher Value-Added Measures

Policymakers are revolutionizing teacher evaluation by attaching greater stakes to student test scores and observation-based teacher effectiveness measures, but relatively little is known about why they often differ so much. Quantitative analysis of thirty schools suggests that teacher value-added measures and informal principal evaluations are positively, but weakly, correlated. Qualitative analysis suggests that some principals give high value-added teachers low ratings because the teachers exert too little effort and are “lone wolves” who work in isolation and contribute little to the school community. The results suggest that the method of evaluation may not only affect which specific teachers are rewarded in the short term, but shape the qualities of teacher and teaching students experience in the long term.

Toward Policy-Relevant Benchmarks for Interpreting Effect Sizes: Combining Effects with Costs.

The common reporting of effect sizes has been an important advance in education research in recent years. However, the benchmarks used to interpret the size of these effects—as small, medium, and large—do little to inform educational administration and policy making because they do not account for program costs. The author proposes an approach to establishing cost-effectiveness benchmarks rooted in an explicit economics-based decision-making framework and assumptions about the decision-making context. To be considered large, the ratio of effects to costs must be at least as large as the ratios for substitute interventions. Evidence related to class size, prekindergarten, and other interventions is discussed to illustrate the calculation of the cost-effectiveness ratios, how the evidence can be used to develop benchmarks, and how the benchmarks can be useful for researchers and policy makers. The development of benchmarks is intended to encourage cost-effectiveness analysis as a standard part of policy analysis, thereby providing more evidence to increase the validity of the benchmarks and, ultimately, improving policy decisions. Recent cost-effectiveness research in health care policy illustrates the potential value of cost-effectiveness benchmarks in education.

Certify, Blink, Hire: An Examination of the Process and Tools of Teacher Screening and Selection

While much has been written about the process of employee selection in other occupations, there has been little discussion on the process and tools of teacher selection and why it occurs as it does. To understand this question, we conduct an extensive literature review in which we compare teacher hiring with hiring in other occupations. We also present findings from a study of school principals and district administrators in a midsized Florida school district. Our results suggest that the screening and selection process in teaching is not much different from occupations that have similar levels of job complexity. A theory emerges from the review and analysis that explains the process and reliance on certain tools in teacher hiring. The theory focuses especially on the costs of various tools and processes, the types and quality of information that come from them, and the distinctive features of teaching as an occupation and schools as organizations.

Why Financial Aid Matters (or Does Not) for College Success: Toward a New Interdisciplinary Perspective

The failure to account for group differences in responsiveness to financial aid is a primary shortcoming of existing higher education research, and compromises the explanatory power of theories and models. Even more importantly, it limits the ability of policymakers and practitioners to achieve the important goals of increased college success and reduced achievement gaps. In this chapter, we identify and explore concepts from social sciences disciplines which hold promise in terms of informing future theoretical developments in the field. Drawing primarily on the work of behavioral economists, and economic sociologists, we discuss concepts such as aversion to risk, work centrality, ambiguity aversion, and consistency theory which may serve to illuminate several unexplained anomalies in prior empirical research. We also describe the methodological problems plaguing the study of financial aid and argue that tests of existing theories, as well as those we propose here, require research that addresses selection bias in who receives aid. We then describe other aspects of future research critical to the testing of these concepts.

Reducing income inequality in educational attainment: : Experimental evidence on the impact of financial aid on college completion

Income inequality in educational attainment is a long-standing concern, and disparities in college completion have grown over time. Need-based financial aid is commonly used to promote equality in college outcomes, but its effectiveness has not been established, and some are calling it into question. A randomized experiment is used to estimate the impact of a private need-based grant program on college persistence and degree completion among students from low-income families attending 13 public universities across Wisconsin. Results indicate that offering students additional grant aid increases the odds of bachelor’s degree attainment over four years, helping to diminish income inequality in higher education.