Dougal Hutchison

How do season of birth and length of schooling affect children's attainment at key stage 1?

Summary A review of previous studies identifies three main hypotheses to explain the general finding that summer‐born children perform less well than their autumn‐born classmates. This article reports the findings of an analysis of the 1991 National Curriculum Assessment data in relation to season of birth. Two hypotheses are explored in an analysis of the results obtained by a sample of around 4,000 six‐ and seven‐year‐olds in maths, science and English. The analysis revealed that there were significant differences between children of different age‐related groups in all three subjects. Although this was partially the result of differences in the age of the children when tested, other factors were found to be related to the achievement of these groups. The findings indicate that both age on starting school and length of schooling are important factors. Children who started school close to the age of four did less well than others. For older children, length of schooling appeared to relate positively to ac...

Sophisticated Tasks in E-Assessment: What Are They and What Are Their Benefits?

This article asserts the importance of e‐assessment. It further suggests that assessment questions and tasks will change substantially as the art of e‐assessment progresses. The article then exemplifies sophisticated e‐assessment tasks, notes taxonomic schemes that have attempted to classify them and seeks to identify aspects of their definition. Next, some key claims for sophisticated e‐assessment tasks are summarised and evaluated. These claims are: (1) sophisticated e‐assessment tasks can be used to assess novel constructs, and (2) sophisticated e‐assessment tasks can be used to address summative and formative assessment purposes. In the final part of the article, issues arising from the article’s findings are discussed and necessary areas for further research are noted.

Adding value in educational research—the marriage of data and analytical power

The last 20 years have seen the development of sophisticated techniques for analysing individual data within a hierarchical context and the growing availability of good datasets to which these techniques can be applied. The modelling of pupil performance controlling for prior attainment has led to a type of analysis commonly titled ‘value-added’. Concern for school factors which affect pupil progress has given rise to ‘school effectiveness’ research. This article outlines the history of these linked movements, mainly with reference to England and Wales, but developments in other countries are also outlined. It discusses some of the objections that have been raised, and comments on possible future directions.

School effectiveness studies using administrative data

Summary This paper presents results of a study of the effectiveness of schools in teaching reading. Some 2,500 pupils in primary schools were given a standard test at ages six and eight, and their progress compared over the period. A number of other school‐level and pupil‐level variables were collected and sieved to see which had a relation with reading progress. After allowing for the measures found to be effective in the preliminary sieving process using multilevel modelling, the measure of school effectiveness thus formed was compared with raw mean scores for schools, and substantial differences were found between the two rankings. The method used gives a fairer comparison of effectiveness between schools than does comparison of the raw mean results, and may be a particularly attractive proposition since it uses data already available from administrative sources.

An evaluation of computerised essay marking for national curriculum assessment in the UK for 11‐year‐olds

This paper reports a comparison of human and computer marking of approximately 600 essays produced by 11-year-olds in the UK. Each essay script was scored by three human markers. Scripts were also scored by the e-rater program. There was a good agreement between human and machine marking. Scripts with highly discrepant scores were flagged and assessed blind by expert markers for characteristics considered likely to produce human–machine discrepancies. As hypothesised, essays marked higher by humans exhibited more abstract qualities such as interest and relevance, while there was little, if any, difference on more mechanical factors such as paragraph demarcation.

Event history and survival analysis in the social sciences II. Advanced applications and recent developments

A previous paper (Hutchison, 1988) in this journal has provided an introduction to the basic concepts of survival and event history analysis, originally developed in medical research, econometrics and engineering, and argued the case for their wider application in the social sciences. This paper introduces some further complications that the researcher is likely to meet, and offers some guidelines for handling problems that arise in applying such methods to the highly complex social situations involved.

Comparing School Systems To Explain Enduring Birth Date Effects: A Response to McDonald (2001).

We are writing in response to the above article published in Volume 31:3 of Compare 2001. The author, Geraldine McDonald, focuses on research into the relative performance of the youngest in the year group (often referred to as ‘summer-born children’) in different countries. Among those in which birth date effects have been noted are Cyprus, Australia, New Zealand, Israel, the USA and the UK. McDonald attempts to explain the persistence of season of birth effects in the UK by referring to the mechanism of ‘selective promotion’. We wish to take issue with her argument, as far as her discussion of the UK is concerned. We understand the three main steps in the author’s argument to be as follows:

Concorde and discord: the art of multilevel modelling

A recent issue of International Journal of Research & Methods in Education (IJRME) contained a challenging article by Stephen Gorard (2007) in which he attacks aspects of current practice in statistical modelling, with particular focus on multilevel modelling (MLM). We believe there is much in his argument which is misleading or misconceived, both at a general and a detailed level, and this article explains where the difficulties arise. We start by a brief overview of some of the basic principles of statistical modelling. We then discuss MLM and some of the criticisms in general. Finally, in the last section before conclusions we detail some of the mis-statements in the Gorard article.

Designing your sample efficiently: clustering effects in education surveys

Background: Education, and information about education, is highly structured: individuals are grouped into classes, which are grouped into schools, which are grouped into local authorities, which are grouped into countries. The degree of similarity among members of a group, such as a school or classroom, is a very important factor in the design and analysis of studies in education. Purpose: The aim of this article is to provide information on this degree of similarity within schools to enable those involved in carrying out surveys of schools to do so most efficiently in terms of resources and minimum disturbance of schools. Sources of data: This paper uses data from 13 studies at primary and secondary level conducted by the National Foundation for Educational Research in England and Wales. Main argument: The degree of similarity among members of a group is measured by two statistics, the intra-cluster correlation and the design effect. The study described here classifies outcomes into a number of categories and estimates intra-cluster correlation and design effect. The relevance of the results to survey design and analysis is discussed, and examples of how to use these are given. Conclusions: The main findings of this study, rather than conclusions as such, are the intra-cluster correlations for each topic category. However, the paper reaches some tentative conclusions about the degree of clustering by topic. Using Hoxs convention (Multilevel analysis: Techniques and applications, Lawrence Erlbaum, London, 2002) for the size of intra-cluster correlations, it was found that the degree of clustering of achievement was high, while ethnic and language variables were highly clustered in secondary but not primary. By contrast, attitudes towards school, educationally relevant home characteristics, and perception of school policies have quite low values of ρ (mean < 0.05), defined as small.

On the Conceptualisation of Measurement Error.

There is a degree of instability in any measurement, so that if it is repeated, it is possible that a different result may be obtained. Such instability, generally described as ‘measurement error’, may affect the conclusions drawn from an investigation, and methods exist for allowing it. It is less widely known that different disciplines, and different workers within a discipline, have different interpretations of the term ‘measurement error’, and that these different interpretations are liable to give rise to different results. This article is an attempt to describe and systematise a number of conceptualisations of measurement error, and the associated concept of randomness, and to show how these work out in practice. Because of the importance of this topic, the aim has been to make the descriptions accessible, avoiding jargon and other technical matters such as equations as far as is feasible.