Cyril J. Weir

Reading in a second language : process, product and practice

PART 1 1. Introduction 2. Preliminaries: What is Reading?PART 2: THE THEORY OF READING 2.1 Models 2.2 Components in Detail 2.3 Comprehension, Skills, Strategies and Styles 2.4 Different Kinds of Reading PART 3: TESTING READING COMPREHENSION (S) 3.1 Introduction: The Limits of Testing 3.2 Task Based Factors 3.3 Text Based Factors 3.4 Test Formats 3.5 Conclusions PART 4: THE TEACHING OF READING 4.1 Teaching and Testing? 4.2 Focus on Metacognitive Strategies 4.3 Focus on Cognitive Strategies and Skills 4.4 What Should Happen in the Reading Classroom? PART 5: FUTURE RESEARCH 5.1 Models and Different Types of Reading 5.2 Reading and the Teaching of Grammar 5.3 The Use of Tests to Investigate Componentiality in Reading Research

Limitations of the Common European Framework for developing comparable examinations and tests

The Common European Framework of Reference (CEFR) posits six levels of proficiency and defines these largely in relation to empirically derived difficulty estimates based on stakeholder perceptions of what language functions expressed by ‘Can-do’ statements can be successfully performed at each level. Though also containing much valuable information on language proficiency and advice for practitioners, in its present form the CEFR is not sufficiently comprehensive, coherent or transparent for uncritical use in language testing. First, the descriptor scales take insufficient account of how variation in terms of contextual parameters may affect performances by raising or lowering the actual difficulty level of carrying out the target ‘Can-do’ statement. In addition, a test’s theory-based validity - a function of the processing involved in carrying out these ‘Can-do’ statements - must also be addressed by any specification on which a test is based. Failure to explicate such context and theory-based validity parameters - i.e., to comprehensively define the construct to be tested - vitiates current attempts to use the CEFR as the basis for developing comparable test forms within and across languages and levels, and hampers attempts to link separate assessments, particularly through social moderation.

Establishing test form and individual task comparability: a case study of a semi-direct speaking test

Examination boards are often criticized for their failure to provide evidence of comparability across forms, and few such studies are publicly available. This study aims to investigate the extent to which three forms of the General English Proficiency Test Intermediate Speaking Test (GEPTS-I) are parallel in terms of two types of validity evidence: parallel-forms reliability and content validity. The three trial test forms, each containing three different task types (read-aloud, answering questions and picture description), were administered to 120 intermediate-level EFL learners in Taiwan. The performance data from the different test forms were analysed using classical procedures and Multi-Faceted Rasch Measurement (MFRM). Various checklists were also employed to compare the tasks in different forms qualitatively in terms of content. The results showed that all three test forms were statistically parallel overall and Forms 2 and 3 could also be considered parallel at the individual task level. Moreover, sources of variation to account for the variable difficulty of tasks in Form 1 were identified by the checklists. Results of the study provide insights for further improvement in parallel-form reliability of the GEPTS-I at the task level and offer a set of methodological procedures for other exam boards to consider.

Can placement tests inform instructional decisions

Studies of placement tests are typically narrowly concerned with their validation as instruments for the efficient grouping of students. They rarely explore the assumption that placement test content can be related to classroom tasks and so inform instructional decisions. This study focuses on a trial version of the Global Placement Test (GPT), a measure of grammatical knowledge being developed for use in language schools worldwide. The central concern is the extent to which the GPT can enable schools to make valid inferences regarding mastery of the grammatical structures to be taught on the relevant courses and hence guide diagnostic intervention. In this study the GPT, together with another test specifically designed to measure knowledge of grammatical items expressing temporal relations in English, was administered to 1070 EFL (English as a Foreign Language) learners studying in the UK, Greece and Japan. The dimensionality of the GPT as a measurement scale was investigated by IRT and factor-analytic methods. The analysis suggests that item difficulty is affected more by item type than by any inherent linguistic difficulty of the element of grammatical competence being tested. The results cast doubt on whether quick grammar-based placement tests, which traditionally emphasize efficiency and practicality, can provide sufficient information on which to base diagnostic mediation or sequencing of instructional materials.

Before the Test Event: A Priori Validity Evidence

There are two aspects to theory-based validity. One concerns a priori evidence collected before the test event, the other a posteriori evidence generated after the test has been administered (Weir 1988a).

Research Methodologies for Exploring the Validity of a Test

There are many excellent general textbooks available on doing research and you will find references at the end of this chapter. To find out more about qualitative and quantitative research design you are referred to them, and references are also given on specific methodologies such as questionnaire design, interview, and verbal protocols, statistical analysis procedures, and discourse analyses of testee performance.

Context Validity in Action

The last decade of the twentieth century saw a general decline in the prestige of psychometric, statistically-driven approaches to testing. In its place there has been a growing interest in the importance of context, in defining domain of use performance conditions and operations.

External Validities in Action

Once a test has been successfully developed, administered and the results shown to be reliable, there still remains the question of what other evidence needs to be generated to demonstrate that it is valid. Two further steps need to be taken. The first involves looking for an external criterion beyond the test in question against which it might be measured.

Language Testing Past and Present

Language tests from the distant past to the present are important historical documents. They can help inform us about attitudes to language, language testing and language teaching when little alternative evidence of what went on in the bygone language classroom remains. Seeing where we have come from also helps us better understand where we are today. The Cambridge ESOL Certificate of Proficiency in English (CPE) has by far the longest track record of any serious EFL examination still in existence, so it is a particularly useful vehicle for researching where we have come from in European approaches to language teaching and testing over the last century. We will trace some significant events in its history to exemplify the developments in the field during that period (see Weir 2003 for a full history of the CPE).

After the Test Event: A Posteriori Validity Evidence

In Quote 2.5, Alderson (1991a) noted how his students were often confused when they came to deal with internal consistency estimates of reliability and also parallel forms of reliability. He argued that these may be equally well regarded as evidence of validity. You may well ask, ‘So, what is reliability? How or does it in fact differ at all from validity?’ We propose to use the term scoring validity as the superordinate for all the aspects of reliability discussed below in line with the growing consensus that it is a valuable part of a test’s overall validity. When referring to specific reliabilities and in quoting the work of others we will retain the traditional labelling.