Jere Confrey

Comparing and Contrasting the National Research Council Report On Evaluating Curricular Effectiveness With the What Works Clearinghouse Approach

This article summarizes the findings of the National Research Council (NRC) report On Evaluating Curricular Effectiveness and examines the reviews in middle grades mathematics undertaken by the What Works Clearinghouse (WWC). The NRC report reviewed and assessed 147 key evaluations of 13 National Science Foundation–supported K–12 mathematics curricula and six commercially generated curricula. The report found that the evaluations overall were not sufficiently robust to permit confident judgments on individual programs, so it instead focused on how to define effectiveness in conducting future evaluations. Effectiveness was defined as “an integrated judgment based on interpretation of a number of scientifically valid evaluations that combine social values, empirical evidence, and theoretical rationales” (NRC, 2004, p. 4). The report introduced a model for curricular evaluation that includes program theory, implementation, and outcome measures, and reviewed three major methodologies found in the literature: content analysis, comparative analysis, and case study. This article then examines the What Works Clearinghouse’s exclusive emphasis on experimental and quasi-experimental designs from the perspective of an author of the NRC report. The two reports agree in recognizing the need for significant improvement in evaluation quality; however, they differ in four areas: standards for individual studies, need for multiple methods, the way to accumulate information across a set of studies, and how to communicate results with the public. This article concludes with a call for focused efforts to address several shared targets needed to make further progress on how to establish curricular effectiveness.

Secondary Teachers’ Statistical Reasoning in Comparing Two Groups

The importance of distributions in understanding statistics has been well articulated in this book by other researchers (for example, Bakker & Gravemeijer, Chapter 7; Ben-Zvi, Chapter 6). The task of comparing two distributions provides further insight into this area of research, in particular that of variation, as well as to motivate other aspects of statistical reasoning. The research study described here was conducted at the end of a 6-month professional development sequence designed to assist secondary teachers in making sense of their students’ results on a statemandated academic test. In the United States, schools are currently under tremendous pressure to increase student test scores on state-developed academic tests. This paper focuses on the statistical reasoning of four secondary teachers during interviews conducted at the end of the professional development sequence. The teachers conducted investigations using the software FathomTM in addressing the research question: “How do you decide whether two groups are different?” Qualitative analysis examines the responses during these interviews, in which the teachers were asked to describe the relative performance of two groups of students in a school on their statewide mathematics test. Preand posttest quantitative analysis of statistical content knowledge provides triangulation (Stake, 1994), giving further insight into the teachers’ understanding.

A Theory of Mathematical Modelling in Technological Settings

A theory of mathematical modelling in education is offered, based on Dewey’s description of inquiry. One aim is that a model provide a mapping between two stages of experience, rather than necessarily a mapping to a particular version of reality; a second aim is that it prepare students for further inquiry and reasoning experience. Two clinical interviews of students engaged in modelling provide examples of progress from an indeterminate to a determinate situation, and of modelling’s potential in differentiated instruction.

Undertaking data analysis of student outcomes as professional development for teachers

The study reports on collaborations with practitioners to examine the results of students’ performances on high stakes tests as a means to strengthen practitioners’ knowledge of probability and statistics and to empower their conduct of investigations on student performance. Four issues are summarized: the development of their statistical reasoning, their understanding of the meaning of and relationships among the concepts of validity, reliability and fairness as applied to testing, their introduction to the history of testing and its relationship to science, society and cultural inequality, and their reports of independent inquiries. Data on performance on pre- and post-tests demonstrate growth in teacher reasoning and in their professionalism in raising important issues about testing

Moving the context of modelling to the forefront: Preservice teachers’ investigations of equity in testing

Prospective math and science teachers were engaged in interpretation and analysis of testing data with the innovative statistical software Fathom™. Armed with a few basic statistical concepts, their ability to model complex issues was correlated with their personal engagement in the process and context at the forefront of the modelling activity.

Epistemology and Modelling — Overview

This section introduces the issues of epistemology, including how to relate modeling and mathematical growth, the implications of complexity, and the limitations of the approach.