Matthew P. Steinberg

Supplemental Educational Services and NCLB: Policy Assumptions, Market Practices, Emerging Issues

The supplemental educational services (SES) provision of No Child Left Behind introduces a federally mandated after-school tutoring intervention in schools that fail to make adequate yearly progress. This article examines market dynamics in relationship to the law’s goals of expanding access to and improving the quality of after-school programming. Analysis of operational and financial data from seven SES providers suggest that relative to smaller local firms, national firms are positioned to capture more of the SES market. Analysis of data on SES provider activity (from 2004 to 2006) within an urban district suggests that firms gaining market share charge higher hourly rates and have larger class sizes. Survey data from 30 state administrators reveal the limited capacity of states to monitor providers. This analysis points to a mismatch between the implementation of SES and the concerns for quality and equity that are claimed as priorities within federal law.

The New Educational Accountability: Understanding the Landscape of Teacher Evaluation in the Post-NCLB Era

In the past five years, teacher evaluation has become a preferred policy lever at the federal, state, and local levels. Revisions to teacher evaluation systems have made teachers individually accountable for student achievement to a greater extent than ever before. We describe and analyze the components, processes, and consequences embedded in new teacher evaluation policies in all fifty states, the twenty-five largest school districts, and Washington, DC. We contextualize these policies by basing our analysis in prior research on teacher evaluation, and examining key comparisons between state and district policies, including their treatment of teachers in tested and untested subjects with career and beginning teachers. We find notable differences in how states and the largest districts have structured evaluation policies for all teachers and, in particular, for early career teachers compared with their more veteran counterparts, and for teachers in nontested grades and subjects compared with those in tested grades and subjects.

Examining Teacher Effectiveness Using Classroom Observation Scores: Evidence from the Randomization of Teachers to Students.

Despite policy efforts to encourage multiple measures of performance in newly developing teacher evaluation systems, practical constraints often result in evaluations based predominantly on formal classroom observations. Yet there is limited knowledge of how these observational measures relate to student achievement. This article leverages the random assignment of teachers to classrooms from the Measures of Effective Teaching (MET) study to identify teacher effectiveness using scores from the Framework for Teaching (FFT) instrument, one of the most widely used classroom observation protocols. While our evidence suggests that teacher performance, as measured by the FFT, is correlated with student achievement, noncompliance with randomization and the modest year-to-year correlation of a teacher’s FFT scores constrain our ability to causally identify effective teachers. Implications for policy and practice are discussed.

Does Teacher Evaluation Improve School Performance? Experimental Evidence from Chicago's Excellence in Teaching Project

Chicago Public Schools initiated the Excellence in Teaching Project, a teacher evaluation program designed to increase student learning by improving classroom instruction through structured principal–teacher dialogue. The pilot began in forty-four elementary schools in 2008–09 (cohort 1) and scaled up to include an additional forty-eight elementary schools in 2009–10 (cohort 2). Leveraging the experimental design of the rollout, cohort 1 schools performed better in reading and math than cohort 2 schools at the end of the first year, though the math effects are not statistically significant. We find the initial improvement for cohort 1 schools remains even after cohort 2 schools adopted the program. Moreover, the pilot differentially impacted schools with different characteristics. Higher-achieving and lower-poverty schools were the primary beneficiaries, suggesting the intervention was most successful in more advantaged schools. These findings are relevant for policy makers and school leaders who are implementing evaluation systems that incorporate classroom observations.

Classroom Composition and Measured Teacher Performance What Do Teacher Observation Scores Really Measure

As states and districts implement more rigorous teacher evaluation systems, measures of teacher performance are increasingly being used to support instruction and inform retention decisions. Classroom observations take a central role in these systems, accounting for the majority of teacher ratings upon which accountability decisions are based. Using data from the Measures of Effective Teaching study, we explore the extent to which classroom composition influences measured teacher performance based on classroom observation scores. The context in which teachers work—most notably, the incoming academic performance of their students—plays a critical role in determining teachers’ measured performance. Furthermore, the intentional sorting of teachers to students has a significant influence on measured performance. Implications for high-stakes teacher accountability policies are discussed.

Does Greater Autonomy Improve School Performance? Evidence from a Regression Discontinuity Analysis in Chicago.

School districts throughout the United States are increasingly providing greater autonomy to local public (non-charter) school principals. In 2005–06, Chicago Public Schools initiated the Autonomous Management and Performance Schools program, granting academic, programmatic, and operational freedoms to select principals. This paper provides evidence on how school leaders used their new autonomy and its impact on school performance. Findings suggest that principals were more likely to exercise autonomy over the school budget and curricular/instructional strategies than over professional development and the schools calendar/schedule. Utilizing regression discontinuity methods, I find that receipt of greater autonomy had no statistically significant impact on a schools average math or reading achievement after two years of autonomy. I do find evidence that autonomy positively affected reading proficiency rates at the end of the second year of autonomy. These findings are particularly relevant for policy makers considering the provision of greater school-based autonomy in their local school districts.

Teachers’ Labor Market Responses to Performance Evaluation Reform: Experimental Evidence from Chicago Public Schools

Traditional teacher evaluation systems have come under scrutiny for not identifying, supporting, and, if necessary, removing low-performing teachers from the classroom. Leveraging the experimental rollout of a pilot evaluation system in Chicago, we find that, while there was no main effect of the pilot on teacher exit, the pilot system increased exit for low-rated and nontenured teachers. Furthermore, teachers who exited were lower performing than those who stayed and those who replaced them. These findings suggest that reformed evaluation systems can induce low-performing teachers to exit schools and may also improve the overall quality of the teacher labor force.

The New Landscape of Educational Privatization in the Era of NCLB.

PRIVATIZATION is a buzz word in education circles. It covers a broad range of activities, initiatives, programs, and policies, including charter schools, vouchers, and the contracting out of services and management. Educational privatization has a long history in the United States. (1) In the past two decades, much media and scholarly attention has been devoted to the educational management industry. Educational management organizations (EMOs) are comprehensive in nature and include companies that manage entire school systems or entire schools. These firms typically assume full responsibility for all aspects of school operations, including administration, teacher training, and such noninstructional functions as building maintenance, food service, and clerical support. Edison Schools, the brainchild of entrepreneur Chris Whittle, is perhaps the best known of the EMOs. However, educational privatization has implications for public schooling far beyond what is evident in the efforts of todays EMOs. The next chapter of educational privatization is being written by firms of a different kind, which have tended to receive much less attention from researchers and the press but cannot be ignored. These are the specialty-service providers. (2) Specialty-service providers contract to fulfill specific educational functions. Their products and services range from software for tabulating and reporting test scores to the design of instructional materials. In contrast to other forms of privatization, such as vouchers, school districts maintain direct control over funds paid to specialty-service providers and, in theory, control the use of those funds through the design of requests for proposals and the establishment of contracts. No Child Left Behind (NCLB) is just the most recent effort in a decades-long national movement to give the private sector a larger role in school reform. However, NCLB is distinctive in that it requires, not simply permits, some local school systems to contract with private providers for services. Across the U.S., test publishers, software companies, and research firms are swarming to take advantage of the revenues made available by NCLB. Such well-established firms as ETS have been joined by a newer breed of providers whose product design and marketing strategies have been informed by the Internet. These later firms have names such as PowerSchool, Brainade, and Orions Mind. Many begin as start-ups and then, once they demonstrate their profitability, are acquired by conglomerates such as publishing houses. Like their counterparts among the EMOs, the firms gaining prominence under the new educational privatization are drawing on political networks, new technologies, venture capital, and government revenues to become major suppliers of services to school systems. Among the accountability measures faced by schools that fail to meet NCLBs specified goals is the requirement that they offer students the chance to receive after-school remedial instruction from private service providers. It is more than a little ironic that, while NCLB puts real teeth into its accountability policies for schools and districts, it offers little guidance or meaningful sanctions for strengthening the accountability of private firms that are increasingly responsible for providing such tutoring. To analyze the role of NCLB as a driver of current developments in the K-12 education market, we examined market-trend data from the education industry and from annual reports (1997-2004) filed with the Securities and Exchange Commission by publicly traded key suppliers. (3) We identified four dominant domains of contracting with specialty-service providers in the K-12 education sector: test development and preparation, data management and reporting, remedial services, and content-specific programming. In investigating these four domains, we collected data on the roles of both governmental and nongovernmental organizations. …

The Impact of the Great Recession on Student Achievement: Evidence from Population Data

The Great Recession was the most severe economic downturn in the United States since the Great Depression. Using newly available population-level achievement data from the Stanford Education Data Archive (SEDA), we estimate the impact of the Great Recession on the math and English language arts (ELA) achievement of all grade 3-8 students in the United States. Employing a difference-in-differences strategy that leverages both cross-district variation in the economic shock of the recession and within-district, cross-cohort variation in school-age years of exposure to the recession, we find that the onset of the Great Recession significantly reduced student math and ELA achievement. Moreover, the recessionary effect on student achievement was concentrated among school districts serving more economically disadvantaged and minority students, indicating that the adverse effects of the recession were not distributed equally among the population of U.S. students. We also find that the academic impact of the recession was more severe for students who were older at the time of first exposure to the recession, compared to their younger counterparts. Finally, the recession’s effects on student achievement were concentrated in districts with the largest reductions in teacher personnel, providing evidence that the effects we observe are driven, in part, by the recession’s negative effects on school resources. We discuss the implications of and potential policy responses to economic shocks that adversely affect student achievement and widen educational inequality.

School Autonomy and District Support: How Principals Respond to a Tiered Autonomy Initiative in Philadelphia Public Schools

ABSTRACT A tiered autonomy policy was recently implemented in Philadelphia, where select principals were granted autonomy to manage school operations while others were promised greater district support to improve school functioning. This article provides evidence on how principals used their autonomy and the extent of district support for non-autonomous principals. Principals granted greater autonomy were more likely to change teacher professional development and curriculum and instructional strategies, while principals with longer tenures and more leadership training were more likely to implement organizational changes. Non-autonomous principals reported a misalignment between school and district priorities and limited district support for improving school functioning.