Robert Coe

School Effectiveness Research: criticisms and recommendations

ABSTRACT Much research on school effectiveness has been characterised by largely overstated claims and poor modelling. School effectiveness research (SER) has tended to define ‘effectiveness’ in terms of a restricted and often inappropriate range of outcomes, to overlook the issue of curriculum alignment, to be limited by the absence of longitudinal data, and it has often been characterised by unsupported assumptions about the homogeneity of school ‘effects’. SER needs to provide justification for the interpretation of ‘effectiveness’ defined as the unexplained part of performance in a statistical model, and a much clearer conception of why certain relationships exist is required. SER has yet to demonstrate the extent to which differences among schools in their ‘effectiveness’ are really caused by identifiable factors within the school and, more importantly, factors within the schools control; evaluations of school improvement interventions are generally unconvincing in this respect. Repeated findings of...

Proof Practices and Constructs of Advanced Mathematics Students

Abstract Proof has traditionally been seen as an important aspect of the advanced mathematics curriculum. The study reported here examines the proof practices and constructs of a group of students educated within a ‘post‐Cockcroft’ mathematics curriculum. It finds that although, in the spirit of the reformed curriculum, students are actively involved in developing mathematical ideas, this is within a restricted mode in which ideas of enquiry have become largely instrumentalised. In particular, validation is primarily and predominantly empirical, and there is little concern for illumination and systematisation.

Effectiveness of Consultation on Student Ratings Feedback: A Meta-Analysis

Consultation on student ratings is recognized as an important strategy to support university teachers in learning from student ratings feedback. However, there is little evidence to suggest which practices and strategies are most important for successful consultation. As the use of student ratings increases, such evidence becomes increasingly important. The authors of this article performed an exploratory meta-analysis to determine the effects of various consultation strategies and to identify the conditions under which consultative feedback is most effective in improving teaching effectiveness. The results indicated that the various approaches to consultation are not equally effective. The authors suggest that only with use of certain consultation strategies might the benefits of support with student ratings feedback be maximized.

School Performance Feedback Systems: Conceptualisation, Analysis, and Reflection

Although there is an international trend to publish and feed back information to schools and teachers, the full complexity of improving schools through school performance feedback is not usually recognised. In this article, “school performance feedback systems” (SPFSs) are first conceptualised, and the factors that have contributed to their popularity are analysed. Next, a framework is presented which includes the factors that are crucial for the use and impact of SPFSs. Finally, the balance sheet is drawn up for the evidence on the process, problems, and impact of SPFSs, and strategies for improving schools through performance feedback are formulated.

Effects of Schooling on Reading Performance, Reading Engagement, and Reading Activities of 15-Year-Olds in England

This article reports the findings of an analysis into the effect of one year’s schooling for 15-year-olds in England on reading performance, reading engagement, and reading activities. The analyses were done on PISA 2000 data by applying a regression discontinuity approach within a multilevel framework. The effect of schooling is estimated as the difference between students from two consecutive grades minus the effect of age. A remarkably modest effect on reading performance was found, and no significant effects were found for the other two measures. The effect on reading performance was found to be somewhat stronger in schools with disadvantaged student populations.

Celebration of the Success of Distributed Research with Schools: The CEM Centre, Durham.

The Curriculum, Evaluation and Management (CEM) Centre has grown in just 25 years to become the largest educational research unit in a UK university. It has influenced schools and teachers in unprecedented numbers and has had a considerable impact on policy and practice both directly and indirectly in the UK and beyond. This article summarises this growth and points to key features of its success. It recounts the philosophical approach of the centre and gives examples of its work to illustrate the way in which it has worked. This includes the development of monitoring systems, the creation of innovative tests, the use of monitoring data in research projects, evaluation and the centres commitment to evidence-based work. The article does not report findings in the manner of many journal articles. Rather, it describes the growth and key features of a remarkably successful centre. It seeks to justify this claim and to outline the manner in which it has been successful.

Comparability of GCSE examinations in different subjects: an application of the Rasch model

The comparability of examinations in different subjects has been a controversial topic for many years and a number of criticisms have been made of statistical approaches to estimating the ‘difficulties’ of achieving particular grades in different subjects. This paper argues that if comparability is understood in terms of a linking construct then many of these problems are resolved. The Rasch model was applied to an analysis of data from over 600,000 candidates who took the General Certificate of Secondary Education (GCSE) examinations in England in 2004. Thirty‐four GCSE subjects were included in the final model, which estimated the relative difficulty of each grade in each subject. Other subjects failed to fit, as did the fail grade, U. Significant overall differences were found, with some subjects more than a grade harder than others, though the difficulty of a subject varied appreciably for different grades. The gaps between the highest grades were on average twice as big as those between the bottom grades. Differential item functioning (DIF) was found for male and female candidates in some subjects, though it was small in relation to variation across subjects. Implications of these findings for various uses of examination grades are discussed.

SCHOOL IMPROVEMENT: REALITY AND ILLUSION

ABSTRACT: School improvement is much sought and often claimed. However, it is questionable whether overall achievement in countries such as the USA or England has improved by any significant amount over thirty years. Several school improvement programmes have been claimed as successful, but evaluations, even where they exist, are generally poor: based on the perceptions of participants, lacking any counterfactual or reporting selectively. Accounts of improvement in individual schools are numerous, but are inevitably selective; the attribution of causality is problematic and knowledge of the conditions under which such phenomena are likely to be replicated is limited. School effectiveness research also has yet to identify specific strategies with clear causal effects. In short, many claims of school improvement are illusory. Nevertheless, there are some improvement strategies that are well-defined, feasible and robustly shown to be effective. In future, we need greater clarity and agreement about what constitutes success. Evaluation must be taken more seriously, and its results treated more critically.

Choosing "in" Schools: Locating the Benefits of Specialisation.

Recent policy in England has suggested that educational outcomes will be raised if schools specialise in particular subjects. In contrast, calls for the reform of 16–19 education have suggested that these outcomes will be improved if students become less specialised in their studies. At present, there is a limited evidence base from which to judge these arguments. In particular, we do not know the extent to which students’ achievements in 16–19 education are higher when they choose subjects which play to their perceived strengths. We also do not know whether students are more likely to choose to study subjects taught by more effective departments. That is, outcomes may be affected by the relative strengths of students or departments in circumstances where there is freedom to choose. In this paper we provide evidence of the existence and strength of these relationships. This evidence suggests that reducing the scope within schools for specialisation or competition will reduce average student attainment and these effects ought to be taken into account when evaluating alternative curriculum policies.

Can feedback improve teaching? A review of the social science literature with a view to identifying the conditions under which giving feedback to teachers will result in improved performance

Abstract In view of the increasingly prominent use in schools and colleges of performance indicators, inspection, appraisal and other forms of performance feedback, it seems important to know what evidence there is that such feedback will produce any improvement in standards, and under what conditions this may be expected to occur. The plausible view that people who receive feedback on a task perform better than those who do not is found to be not universally supported by the empirical evidence. Theories which try to account for the effects of feedback on performance are limited in scope, too vague to be readily operationalized or to enable predictions to be made, and/or supported by only some of the available evidence. An attempt is made to identify the significant variables which mediate the effects of feedback on performance. Important characteristics of the task include its complexity, the balance between the demands of effort and ability, and the availability of other information or instruction. Sign...