Marian Sainsbury

Fallback in attainment on transfer at age 11: evidence from the Summer Literacy Schools evaluation

Summary Summer Literacy Schools were introduced as a pilot scheme by the Government in the summer of 1997 in an attempt to improve childrens literacy skills at the age of 11, the time of transfer from primary to secondary education. The initiative was evaluated by collecting the pupils’ results from the national test taken in May and comparing these with results of a similar test administered in September. Results for a control group who had not attended summer schools were also analysed. The analysis revealed that the scores of both groups declined significantly between the pre‐test and the post‐test, and no significant difference in the extent of the decline was found between the summer school pupils and the control group. Further research is needed on the possibility that the transition from primary to secondary school is associated generally with a significant decline in attainment.

Multilevel Analysis of the Key Stage 1 National Curriculum Assessment Data in 1995

Abstract In a follow‐up to previous work (Schagen, 1994), results from Key Stage 1 National Curriculum tests in 1995 have been analysed using multilevel modelling. The analysis estimates the effects of different background variables, at both the pupil and school levels. Some investigation of the influence of reading attainment on mathematics test scores, over and above the effect of general mathematical attainment, was carried out. The results showed the significant effects of certain background variables, including sex, age, nursery education and ethnic background, which persisted even when controlling for Teacher Assessment levels. Mathematics tests scores were shown to be related to reading levels, even when teacher‐assessed mathematical attainment was taken into account.

Level Descriptions in the National Curriculum: what kind of criterion referencing is this?

ABSTRACT The National Curriculum has, since its inception, been described as a criterion‐referenced assessment system, in which attainments are described in terms of statements about what pupils can do, rather than in terms of comparisons with other pupils. Since 1995, the structure of the National Curriculum assessment criteria has been changed to ‘level descriptions’, continuous prose paragraphs describing complex attainments. The theoretical underpinnings for this structure have not, however, been articulated. We adopt a framework for analysis, in which assessment is essentially referenced to an educational construct. Educational constructs are abstract and complex notions of valued educational outcomes. Derived from these constructs are assessment domains, which are more specific statements of what is to be assessed. A survey of the literature of the criterion‐referenced testing tradition reveals a range of approaches to defining these domains, varying from very precise behavioural specifications to b...

Criterion Referencing and the Meaning of National Curriculum Assessment

Criterion-referenced assessment has made promises that it is unable to keep. The idea that a criterion-referenced test may afford a clear and direct interpretation in terms of exactly which tasks an examinee can perform is unattainable for the kinds of learning promoted in complex curricula, such as the National Curriculum in England and Wales. However, examining more carefully the origin of these claims suggests that they reflect a particularly narrow view of criterion referencing, founded on some dubious assumptions. A reanalysis of the notion of criterion referencing shows that there are advantages to adopting an approach that references assessment outcomes to an underlying educational construct, not only in clarifying how those outcomes should be interpreted, but also in terms of manageability. The introduction of level descriptions for National Curriculum assessment is discussed in relation to this approach.

Measuring standards in primary English: the validity of PIRLS—a response to Mary Hilton

Hilton (2006) criticises the PIRLS (Progress in International Reading Literacy Study) tests and the survey conduct, raising questions about the validity of international surveys of reading. Her criticisms fall into four broad areas: cultural validity, methodological issues, construct validity and the survey in England. However, her criticisms are shown to be mistaken. Her claim of forced unidimensionality in the tests is not supported by statistical analyses and her claims of cultural strangeness are contradicted by the involvement of all the countries involved. She is concerned about linguistic diversity but this is actually reflected in the ways countries organise their surveys. Finally, Hilton suggests that the English sample was biased, but fails to recognise the stringent sampling requirements or the monitoring roles of external assessors and the sampling referee. A careful study of the evidence concerning PIRLS shows that it is actually a fair and robust measure of reading attainment in different co...

An investigation of hierarchical relationships in children’s literacy attainments at baseline

This paper describes one element of the research basis for the Baseline Assessment Scales now published for optional use in baseline schemes nationally (SCAA, 1997a). The aim in developing these scales was to provide a range of criteria that would allow almost all children to show some attainment, whilst also acknowledging the attainments of the most able. For this purpose, four-point scales were required. The first point would be attainable by over 80 per cent of children in their first term of school, and the fourth by only 20 per cent or fewer, with two intermediate points. Four such scales were trialled for reading, and one for writing. As part of the trial analysis, an investigation was undertaken into the hierarchies amongst the items on each scale – that is, to what extent was it possible for a child to attain a more difficult item, whilst failing an easier item on the same scale? A ‘coefficient of dependency’ was calculated for each pair of items on each scale. The percentages of children achieving each item are reported, and the strength of the hierarchies amongst them. This provides some evidence as to the interrelationships between children’s literacy attainments at the start of school.

Issues in the Evaluation of Standard Assessment Tasks and the Reliability of National Curriculum Assessment

Important issues in the reliability of National Curriculum assessment were raised by an earlier paper by Davies & Brember, but unfortunately not supported by a valid analysis of the data they collected from five primary schools in 1991. This paper aims to clarify some of those issues and to point out the lack of comparability between the two measures which were compared. Suggestions for more valid analysis are given.