Finbarr C. Sloane

Issues in High-Stakes Testing Programs

The debate over high-stakes testing programs plays out daily in newspapers, on TV, and in the business, education, legal, political, and research communities. This article examines some of the issues at the heart of this debate. Four main areas are covered: the types of tests used, the effects on student motivation and morale, the degree of alignment between the test and the curriculum, and the distinction between assessment of learning and assessment for learning. The article concludes by highlighting the need for teacher input in crafting testing programs that maximize benefits in each of these areas.

Exploring Modeling Aspects of Design Experiments

In this article the authors use the process of model building (model formulation, fit, and validation) in applied settings to raise pertinent questions about design experiment (DE) methodology. We argue that the DE work presented in this issue highlights features of model formulation and local validation, but does not discuss model fitting or broader models of validation. This article marks out key areas for the DE community to address and concludes by positing that the concept of artifact failure in design research may be a more appropriate area of concern when designing an artifact (whether the artifact is a learning process or a software product). DE research is relatively new as an educational research method (Brown, 1992; Collins, 1992). We believe that DE researchers and the more general research methodology communities must work together to fully evaluate and reap the potential rewards of this developing research method.

Randomized Trials in Mathematics Education: Recalibrating the Proposed High Watermark

The author reviews the recommendations in Foundations for Success: The Final Report of the National Mathematics Advisory Panel (2008) and agrees that a rebalancing of mathematics education research is timely and necessary, but questions whether randomized trials of small experimental studies and large field studies, without a clearer framing of the needed continuum of studies, can adequately rebalance the portfolio and address the Panel’s “what works” questions. He offers one listing of the possible phases of research required to support high-quality causal inference in mathematics education as a way to foster continued debate about the ease of moving a model of research that works in one domain (drug trials) into the forced service of another intellectual domain (education).

Educational research and the problems of practice

Abstract In this paper we use the lens of an emerging research method in education called “design research” to address questions raised by Sugrue and Uí Thuama (1994) about the lack of strong linkages between research and classroom practice in Irish education. We provide context for this question by contrasting it with similar discussions in the US and UK about research quality in education. Next we elaborate a framework from one member of the design research community in educational research (Bannan‐Ritland, 2003) to show why some varieties of educational research may have little to offer practice at least directly and in the short term. Finally, we offer insight from Lesson Study in Japan (Lewis, 2002) as one possible solution to the problem of research and practice alignment.

Value-added models of teacher and school effectiveness in Ireland: wise or otherwise?

The government of Ireland, like many European countries, is currently under severe pressure from external forces to grow the economy. One possible way to maintain and grow its economy is through the production of a highly educated and globally competitive workforce. In an effort to develop such a workforce, the government, through the Department of Education, is considering ways to increase accountability in its schools. This paper examines value-added accountability systems used in the USA and raises critical questions about their perceived value for Irish schools.

A Review of Three Large-Scale Datasets Critiquing Item Design, Data Collection, and the Usefulness of Claims

Issues of validity and usefulness of three large-scale longitudinal data sets are reviewed in this chapter. The Trends in International Mathematics and Science Study (TIMSS), the National Assessment of Educational Progress (NAEP), and the Educational Longitudinal Study of 2002 (ELS:2002) are compared and contrasted with respect to differences in sampling frame, internal and external validity, and especially construct validity of assessment items. Conclusions about the usefulness of large-scale secondary data analysis show that the reviewed assessments have been critical for determining inequities of opportunity for gender, ethnicity, socioeconomic status, and across national boundaries. They have also been useful for researchers examining the effectiveness of curricular policy on student learning. Moreover, some stakeholders have used the results as evidence that a nation’s future GDP is predicted by the outcome on TIMSS, and that students need more mathematical knowledge and skills to compete in a world that has an ever increasing rate of technological expansion. Though longitudinal, the duration of the studies presents a problem, as none follow students’ mathematical abilities or development for any length of time (e.g., early childhood into adulthood), and few studies from large-scale assessments shed light onto the kinds of pedagogy or curricular tasks that positively impact student learning. Lastly, threats to validity for large-scale studies are critiqued, and shown to be underreported in the literature.

HLM in cluster-randomised trials – measuring efficacy across diverse populations of learners

We describe the application of Hierarchical Linear Modelling (HLM) in a cluster-randomised study to examine learning algebraic concepts and procedures in an innovative, technology-rich environment in the US. HLM is applied to measure the impact of such treatment on learning and on contextual variables. We provide a detailed description of such methods, methodically analysing nested classroom data with respect to various outcome measures through HLM.