Kathleen Lynch

Validating Arguments for Observational Instruments: Attending to Multiple Sources of Variation

Measurement scholars have recently constructed validity arguments in support of a variety of educational assessments, including classroom observation instruments. In this article, we note that users must examine the robustness of validity arguments to variation in the implementation of these instruments. We illustrate how such an analysis might be used to assess a validity argument constructed for the Mathematical Quality of Instruction instrument, focusing in particular on the effects of varying the rater pool, subject matter content, observation procedure, and district context. Variation in the subject matter content of lessons did not affect rater agreement with master scores, but the evaluation of other portions of the validity argument varied according to the composition of the rater pool, observation procedure, and district context. These results demonstrate the need for conducting such analyses, especially for classroom observation instruments that are subject to multiple sources of variation.

Understanding the development of flexibility in struggling algebra students

The ability to flexibly solve problems is considered an important outcome for school mathematics and is the focus of this paper. The paper describes the impact of a three-week summer course for students who struggle with algebra. During the course, students regularly compared and contrasted worked examples of algebra problems in order to promote flexible use of solution strategies. Assessments were designed to capture both knowledge and use of multiple strategies. The students were interviewed in order to understand their rationales for choosing particular strategies as well as their attitudes toward instruction that emphasized multiple strategies. Findings suggest that students gained both knowledge of and appreciation for multiple strategies, but they did not always use alternate strategies. Familiarity, understandability, efficiency, and form of the problem were all considerations for strategy choice. Practical and theoretical implications are discussed.

Teachers’ Views About Multiple Strategies in Middle and High School Mathematics

Despite extensive scholarship about the importance of teaching mathematics with multiple strategies in the elementary grades, there has been relatively little discussion of this practice in the middle and high school levels or in the context of introductory algebra. This article begins our exploration of this practice by addressing the following questions: (1) What do middle and high school Algebra I teachers describe as the advantages of instruction that includes a focus on multiple strategies?; and (2) What disadvantages to this practice do these teachers describe? Our analysis, based on the data from interviews (N= 13) and surveys (N= 79) conducted with experienced middle and secondary mathematics teachers, indicates that middle and secondary math teachers’ reported views surrounding multiple strategies appear to differ in important ways from those typically associated with teaching with multiple strategies in the elementary grades.

Resources for Teaching

Policymakers and researchers have for many years advocated disparate approaches to ensuring teachers deliver high-quality instruction, including requiring that teachers complete specific training requirements, possess a minimum level of content knowledge, and use curriculum materials and professional development resources available from schools and districts. In this paper, we investigate the extent to which these factors, which we conceptualize as resources for teaching, predict instructional quality in upper elementary mathematics classrooms. Results show that teachers’ mathematical knowledge and their district context explained a moderate share of the variation in mathematics-specific teaching dimensions; other factors, such as teacher experience, preparation, non-instructional work hours, and measures of the school environment, explained very little variation in any dimension.

Relationships between Observations of Elementary Mathematics Instruction and Student Achievement: Exploring Variability across Districts

Much debate surrounding teacher quality has focused on students’ standardized test scores, but recent federal and state initiatives have emphasized the use of multiple measures to evaluate teacher quality, including classroom observations. In this study, we explore differences across school districts in the relationship between student achievement outcomes and the observed quality of teachers’ instruction. Using data from 298 elementary mathematics teachers in five urban US districts, we examine relationships between teachers’ performance on the Mathematical Quality of Instruction observation instrument and their students’ scores on both state standardized and researcher-developed tests. We find that these relationships differ across school districts. We explore the extent to which differences in skills and expectations for students across tests may explain this variability. An improved understanding of the relationship between classroom observations and student tests may help districts to better support teachers in developing their instructional effectiveness.

Effects of a Summer Mathematics Intervention for Low-Income Children: A Randomized Experiment

Prior research suggests that summer learning loss among low-income children contributes to income-based gaps in achievement and educational attainment. We present results from a randomized experiment of a summer mathematics program conducted in a large, high-poverty urban public school district. Children in the third to ninth grade (N = 263) were randomly assigned to an offer of an online summer mathematics program, the same program plus a free laptop computer, or the control group. Being randomly assigned to the program plus laptop condition caused children to experience significantly higher reported levels of summer home mathematics engagement relative to their peers in the control group. Treatment and control children performed similarly on distal measures of academic achievement. We discuss implications for future research.