Gordon Stobart

Gender Bias in Examinations: how equal are the opportunities?

Abstract Recent analyses of public examination results, particularly in the GCSE, have demonstrated significant gender differences in both entry patterns and outcomes. This generally involves better results for girls in subjects other than mathematics and some sciences. The article discusses whether examination boards should work for equal outcomes given that the assessment techniques used in examinations may differentially affect girls’ and boys’ performances, as evidenced by objective tests and coursework. We argue that the type of assessment is only one factor and that the different experiences and expectations of both pupils and teachers also play significant parts. Equal outcomes should not therefore be contrived simply by manipulation of assessment techniques. What may be required is more sensitivity to gender related differences in approach and experiences. How much the recent changes to national curriculum assessment will take account of these factors will need further monitoring.

Fairness in multicultural assessment systems

This paper is about fairness (equity) in large‐scale assessment systems within multicultural societies. It makes the key assumptions that fairness is fundamentally a sociocultural, rather than a technical, issue and that fair assessment cannot be considered in isolation from both the curriculum and the educational opportunities of the students. Equity is defined as a qualitative concern for what is just. This involves, but is not the same as, equality of opportunity and of outcome. In relation to large‐scale assessment four topics are addressed: the nature of the assessment system; recognizing experiences of different groups; cultural diversity; and monitoring group performance. The conclusion is that, while we can never achieve fair assessment, we can make it fairer. At the heart of this improvement process is openness about design, constructs and scoring which brings out into the open the values and biases of the test design process.

Determining validity in national curriculum assessments

Background: Validity is a central concern in any assessment, though this has often not been made explicit in the UK assessment context. This article applies current validity theorising, largely derived from American formulations, to national curriculum assessments in England. Purpose: The aim is to consider validity arguments in relation to the multiple purposes which national curriculum assessments serve. This is because validity is now understood in terms of how the test results are interpreted and used. Multiple purposes mean multiple uses of the results, each requiring justification. Sources of evidence: A validity framework for national curriculum assessments is presented which is organised around purposes, fitness-for-purpose, reliability and the interpretation and impact of the results. Current tests are evaluated against this framework. Main argument and conclusions: The analysis finds the national test regime successfully meets some of the common threats to validity. The main concerns are about the multiple high-stakes purposes for which the tests are used and their impact on teaching and learning. Some alternative approaches are then considered, which could reduce the burden of interpretation which the tests currently bear.

Assessment and learning: fields apart?

Abstract Educational assessments define what aspects of learning will formally be given credit and therefore have a huge impact upon teaching and learning. Although the impact of high-stakes national and international assessments on teaching and learning is considered in the literature, remarkably, there is little research on the connection between theories of learning and educational assessments. Given the voluminous assessment that takes place annually in systematic ways in most many nations, it is surprising that more has not been gained from these assessments in the development of theories of learning and vice versa. In this article, we consider both theories of learning and assessment and draw the main message of the article, that if assessments are to serve the goals of education, then theories of learning and assessment should be developing more closely with each other. We consider fundamental aspects of assessment theory, such as constructs, unidimensionality, invariance and quantifiability, and in doing so, we distinguish between educational and psychological assessment. Second, we show how less traditionally considered cases of (a) international assessments and (b) Assessment for Learning affect student learning. Through these cases we illustrate the otherwise somewhat theoretical discussion in the article. We argue that if assessment is to serve the learning goals of education, then this discussion on the relationship between assessment and learning should be developed further and be at the forefront of high-stakes, large-scale educational assessments.

High-stakes testing–value, fairness and consequences

High-stakes testing has been with us for over two thousand years and is steadily increasing both in scale and range. This special issue considers some of the main uses of these tests (a term used l...

Teachers’ experiences of autonomy in Continuing Professional Development : Teacher Learning Communities in London and Hong Kong

This paper examines teachers’ experiences of autonomy as they undertook Continuing Professional Development (CPD) in the form of Teacher Learning Communities (TLCs) to develop Assessment for Learning (AfL). Participant teacher interview data were used from two parallel TLC projects, one in Hong Kong and one in London, UK. Autonomy was defined in terms of taking initiatives, acting independently and making critical inquiries. Links between autonomy and effective CPD were argued and the findings indicated that some teachers in both the projects experienced limited autonomy in terms of taking initiatives and being critical, although several described a new sense of professional independence as they attended TLCs. Their limited autonomy looked likely to inhibit the long-lasting impact of the projects, although one TLC in London seemed likely to shape teachers’ AfL principles and practices more profoundly because attendance was voluntary and egalitarian.

Fairness in Assessment

Fairness is a concept for which definitions are important, since it is often interpreted in too narrow and technical a way. We set fairness within a social context and look at what this means in relation to different groups and cultures. Similarly, we are using educational assessment in a more inclusive way than is often the case; we include tests, examinations, teachers’ judgments or evaluations (‘assessment’ in the United Kingdom) of student performance. We then explore bias in measurement and how it relates to validity, as well as the broader concept of equity. Finally, three examples of approaches to ensure fairness are given. We argue that 21st-century assessment will need to take ever more account of the social contexts of assessment and to continue the movement away from seeing fairness simply as a technical concern with test construction. Fairness in assessment involves both what precedes an assessment (for example, access and resources) and its consequences (for example, interpretations of results and impact) as well as aspects of the assessment design itself.

On the supranational spell of PISA in policy

Abstract Background: PISA results appear to have a large impact upon government policy. The phenomenon is growing, with more countries taking part in PISA testing and politicians pointing to PISA results as reasons for their reforms. Purpose: The aims of this research were to depict the policy reactions to PISA across a number of jurisdictions, to see whether they exhibited similar patterns and whether the same reforms were evident. Sources of evidence: We investigated policy and media reactions to the 2009 and 2012 PISA results in six cases: Canada, China (Shanghai), England, France, Norway and Switzerland. Cases were selected to contrast high-performing jurisdictions (Canada, China) with average performers (England, France, Norway and Switzerland). Countries that had already been well reported on in the literature were excluded (Finland, Germany). Design and methods: Policy documents, media reports and academic articles in English, French, Mandarin and Norwegian relating to each of the cases were critically evaluated. Results: A policy reaction of ‘scandalisation’ was evident in four of the six cases; a technique used to motivate change. Five of the six cases showed ‘standards-based reforms’ and two had reforms in line with the ‘ideal-governance’ model. However, these are categorisations: the actual reforms had significant differences across countries. There are chronological problems with the notion that PISA results were causal with regard to policy in some instances. Countries with similar PISA results responded with different policies, reflecting their differing cultural and historical education system trajectories. Conclusions: The connection between PISA results and policy is not always obvious. The supranational spell of PISA in policy is in the way that PISA results are used as a magic wand in political rhetoric, as though they conjure particular policy choices. This serves as a distraction from the ideological basis for reforms. The same PISA results could motivate a range of different policy solutions.

The Key Stage 3 Strategy: what kind of reform is this?

The Key Stage 3 Strategy is a complex and evolving government strategy intended to improve the education of 11‐ to 14‐year‐olds in England. This paper provides a snapshot of its development by early 2004, drawing on our evaluation of the pilot phase which began in 2000. The evolution of the Strategy, with its phased introduction of both subject and cross‐curricular strands, is described. We then examine the policy intentions and evidence base which underpin it before considering what the Strategy offers in terms of curricular and pedagogical reform. While the Strategy seeks to improve schooling for 11‐ to 14‐year‐olds, it has struggled to offer a coherent approach, though recent changes in emphasis may help. We conclude that a more radical approach to the learning needs of 11‐ to 14‐year‐olds is needed and draw evidence from international trends in middle years education.

Large-scale implementation of Assessment for Learning

The publication of Black and Wiliam’s (1998) ‘Assessment and classroom learning’ in this journal and their widely disseminated pamphlet ‘Inside the black box: raising standards through classroom assessment’ initiated an international implementation of what is now generally known as Assessment for Learning. Its take-up has been described as a ‘research epidemic’ which in a relatively few years has ‘feverishly spread into every discipline and professional field’ (Steiner-Khamsi, 2004, p. 2). This is reflected in the range of contributors to this issue in which large-scale initiatives from eight different counties are reported. A recent review of the state of the field (Baird, Hopfenbeck, Newton, Stobart, & Steen-Utheim, 2014) found 907 peer-reviewed articles and 481 conference proceedings on Assessment for Learning and formative assessment. Fewer than 10 of these studies could be described as large-scale, the vast majority being case studies in one or two schools, with relatively few students. However, Assessment for Learning has been taken up widely at national and regional levels though we have as yet relatively little evidence of how successful these policies have been. This Special Issue is therefore a timely attempt to draw together research that has looked at large-scale implementation, either in the form of national policy or involving relatively large numbers of schools and teachers. In looking at the implementation of Assessment for Learning policies at the national level, we become aware of the importance of the social and educational contexts into which they are being introduced. A common theme here is the impact of summative assessment and accountability measures on the implementation of Assessment for Learning. In the opening article, María Teresa Flórez Petour uses her case study of assessment policy to demonstrate how the complex interaction of systems, ideologies and history impacts on the take-up and implementation of Assessment for Learning in Chile. Given this complexity how do we implement Assessment for Learning practices on a large scale, given many schools and teachers may be conscripts rather than enthusiastic volunteers? It is not simply a question of making clear what Assessment for Learning involves, we need mechanisms for supporting its introduction. This raises questions about how assessment reform can be more effectively implemented. Several of the large-scale initiatives adopt an explicit model of implementation. The studies from Sweden (Jonsson, Holmgren and Lundahl), Canada (DeLuca, Klinger, Piper and Woods), the USA (Wylie and Lyon) and Scotland (Hayward) utilise a variety of implementation models drawn from research. In the case of Norway (Hopfenbeck, Tolo and Flórez Petour), the dissemination model, a four-year rolling programme of working from the policy centre with volunteer groups, is less theoretically explicit but relies on continuous adaptation. The study of a national initiative in Trinidad and Tobago (De Lisle) demonstrates how the