Richard Correnti

Opening Up the Black Box: Literacy Instruction in Schools Participating in Three Comprehensive School Reform Programs

This study examines patterns of literacy instruction in schools adopting three of America’s most widely disseminated comprehensive school reform (CSR) programs (the Accelerated Schools Project, America’s Choice, and Success for All). Contrary to the view that educational innovations seldom affect teaching practices, the study found large differences in literacy instruction between teachers in America’s Choice schools and comparison schools and between teachers in Success for All schools and comparison schools. In contrast, no differences in literacy teaching practices were found between teachers in Accelerated Schools Project schools and comparison schools. On the basis of these findings and our knowledge of the implementation support strategies pursued by the CSR programs under study, we conclude that well-defined and well-specified instructional improvement programs that are strongly supported by on-site facilitators and local leaders who demand fidelity to program designs can produce large changes in teachers’ instructional practices.

Using Teacher Logs to Measure the Enacted Curriculum: A Study of Literacy Teaching in Third‐Grade Classrooms

In this article we examine methodological and conceptual issues that emerge when researchers measure the enacted curriculum in schools. After outlining key theoretical considerations that guide measurement of this construct and alternative strategies for collecting and analyzing data on it, we illustrate one approach to gathering and analyzing data on the enacted curriculum. Using log data on the reading/language arts instruction of more than 150 third‐grade teachers in 53 high‐poverty elementary schools participating in the Study of Instructional Improvement, we estimated several hierarchical linear models and found that the curricular content of literacy instruction (a) varied widely from day to day, (b) did not vary much among students in the same classroom, but (c) did vary greatly across classrooms, largely as the result of teachers’ participation in 1 of the 3 instructional improvement interventions (Accelerated Schools, America’s Choice, and Success for All) under study. The implications of these findings for future research on the enacted curriculum are discussed.

Studying Reading Instruction With Teacher Logs: Lessons From the Study of Instructional Improvement

This article describes some of the conceptual and methodological issues that arise when researchers use teacher logs to measure classroom instruction. Data and examples come from the Study of Instructional Improvement, which used teacher logs to study patterns of literacy instruction in schools implementing three comprehensive school reforms. Over the course of this study, more than 75,000 logs were collected from nearly 2,000 teachers in Grades 1 through 5. This article discusses why teacher logs were chosen as the data collection strategy, various psychometric issues associated with their use, and some of the substantive findings that emerged as part of the study.

An Empirical Investigation of Professional Development Effects on Literacy Instruction Using Daily Logs

The author examined the effects of professional development (PD) on literacy instruction using 75,689 lessons from 1,945 classrooms in 112 schools participating in the Study of Instructional Improvement. Stratifying over 94 pretreatment covariates, including prior instruction, the results revealed the importance of PD as a lever for changing teacher practice. Teachers receiving intense PD in comprehension offered 10% more comprehension instruction than teachers not receiving intense PD. Similarly, teachers receiving intense PD in writing offered 13% more writing instruction and had students write 12% more text than other teachers. Furthermore, these estimates may be lower bounds, because they were estimated over a single year and because there was demonstrated potential for additional influences of the school context.

The Quality of Writing Tasks and Students’ Use of Academic Language in Spanish

This study investigates the quality of the writing tasks assigned to native Spanish speakers in bilingual (Spanish-English) contexts, and the relationship between task quality and students’ use of an academic register in their native language. Fifty-six language arts tasks were collected from 26 grade 4 and 5 teachers, and four student writing samples were collected in response to each task (N = 224). Multilevel modeling revealed that variation in students’ use of key features of academic language in their writing was associated with the cognitive demand of writing tasks. Findings suggest that students’ opportunities to respond to challenging tasks when writing in their native language are rare and that the rigor of writing tasks may relate to students’ production and development of academic language.

Conceptual, Methodological, and Policy Issues in the Study of Teaching: Implications for Improving Instructional Practice at Scale

Education research and policy are increasingly converging around the notion that improving education outcomes system-wide will entail a large-scale effort to improve instruction inside the classroom. Instruction is one of the critical factors mediating the relationship between education policy and student outcomes, and thus an accurate sense of what goes on inside classrooms is needed to understand not only what works? in education, but also how? (Raudenbush & Sadoff, 2008). With a renewed focus on the critical role of instruction in classrooms as a determinant of student achievement, educational research efforts seeking to isolate the components of effective teachers (e.g., the Gates Foundation’s Measures of Effective Teaching Study) and effective teaching (e.g., Hiebert & Morris, 2012, described this important distinction) are again becoming prominent. In parallel, many districts and states are developing systems of teacher evaluation (Baker et al., 2010) that include among the key indicators measures of instruction (or classroom practice more generally) intended to help open the black box of the classroom. Although instruction is at the center of most conceptual models of educational quality and effectiveness, research and policy efforts often have focused on educational outcomes, paying less attention to examining educational processes, both as mediators and as important outcomes themselves. It is important to note that this is rather unlikely to result from competing ideas about whether gaining knowledge about teaching is important. Rather, it reflects the challenges facing researchers interested in rigorously studying instructional practice and in organizing

Automatic Scoring of an Analytical Response-To-Text Assessment

In analytical writing in response to text, students read a complex text and adopt an analytic stance in their writing about it. To evaluate this type of writing at scale, an automated approach for Response to Text Assessment RTA is needed. With the long-term goal of producing informative feedback for students and teachers, we design a new set of interpretable features that operationalize the Evidence rubric of RTA. When evaluated on a corpus of essays written by students in grades 4-6, our results show that our features outperform baselines based on well-performing features from other types of essay assessments.

Examining Implementation Fidelity in America's Choice Schools: A Longitudinal Analysis of Changes in Professional Development Associated with Changes in Teacher Practice.

The research on professional development (PD) consists mostly of studies utilizing cross-sectional data. We examined effects of change in school-level PD on change in teachers’ practice longitudinally. Using survey reports from 1,722 teachers in 31 schools implementing a popular comprehensive school reform (America’s Choice), we found that although schools were implementing the same reform model, they varied in their implementation of PD. Teachers successfully changed their practice toward reform-aligned instructional goals only in schools demonstrating high growth in providing reform-aligned PD (both in the content and process of their PD). Our findings have implications for understanding interventions in schools and for thinking about the rigor of PD effects on teaching.

Using Instructional Logs to Identify Quality in Educational Settings.

When attempting to identify educational settings that are most effective in improving student achievement, classroom process (that is, the way in which a teacher interacts with his or her students) is a key feature of interest. Unfortunately, high-quality assessment of the student-teacher interaction occurs all too infrequently, despite the critical role that understanding and measuring such processes can play in school improvement. This article discusses the strengths and weaknesses of two common approaches to studying these processes-direct classroom observation and annual surveys of teachers-and then describes the ways in which instructional logs can be used to overcome some of the limitations of these two approaches when gathering data on curriculum content and coverage. Classroom observations are expensive, require extensive training of raters to ensure consistency in the observations, and because of their expense generally cannot be conducted frequently enough to enable the researcher to generalize observational findings to the entire school year or illuminate the patterns of instructional change that occur across the school year. Annual surveys are less expensive but often suffer from self-report bias and the bias that occurs when teachers are asked to retrospectively report on their activities over the course of a single year. Instructional logs offer a valid, reliable, and relatively cost-effective alternative for collecting detailed information about classroom practice and can overcome some of the limitations of both observations and annual surveys.

Measuring Reading Instruction With Teacher Logs

The authors argue that the criticisms of their earlier article on teacher logs (Educational Researcher, March 2009) by Smagorinsky and Willis (this issue of Educational Researcher) do not address, much less undermine, the evidence they presented as part of their validation argument about the teacher logs. Moreover, they argue that their method for studying classrooms is not nearly as incommensurate with Smagorinsky’s and Willis’s methods as those authors’ arguments seem to imply. The authors of this rejoinder see the main differences between themselves and their critics as being around the notion of “consequential” validity, an issue they discuss at the end of this article.