Thom Hudson

The Alternatives in Language Assessment

Language testing differs from testing in other content areas because language teachers have more choices to make. The purpose of this article is to help language teachers decide what types of language tests to use in their particular institutions and classrooms for their specific purposes. The various kinds of language assessments are classified into three broad categories: (a) selected-response assessments (including true-false, matching, and multiple-choice assessments); (b) constructedresponse assessments (including fill-in, short-answer, and performance assessments); and (c) personal-response assessments (including conference, portfolio, and self- or peer assessments). For each assessment type, we provide a clear definition and explore its advantages and disadvantages. We end the article with a discussion of how teachers can make rational choices among the various assessment options by thinking about (a) the consequences of the washback effect of assessment procedures on language teaching and learning, (b) the significance of feedback based on the assessment results, and (c) the importance of using multiple sources of information in making decisions based on assessment information.

Teaching second language reading

This handbook will be useful for both beginning and experienced teachers who want to improve their practical strategies in teaching second language reading and their understanding of the reading process. The book examines a variety of approaches from classrooms and research that are used in teaching reading, and explores teaching methods focused on strategies. Teachers are encouraged to think about their own beliefs and opinions on the nature of reading and to examine their own personal reading activities.

Examinee abilities and task difficulty in task-based second language performance assessment

This article summarizes findings from investigations into the development and use of a prototype English language task-based performance test. Data included performances by 90 examinees on 13 complex and skills-integrativetasks, a priori estimations of examinee proficiency differences, a priori estimations of task difficulty based on cognitive processing demands, performance ratings according to task-specific as well as holistic scales and criteria, and examinee self-ratings. Findings indicated that the task-based test could inform intended inferences about examinees’ abilities to accomplish specific tasks as well as inferences about examinees’ likely abilities with a domain of tasks. Although a relationship between task difficulty estimates and examinee performances was observed, these estimates were not found to provide a trustworthy basis for inferring examinees’ likely abilities with other tasks. These findings, as well as study limitations, are further discussed in light of the intended uses for performance assessment within language education, and recommendations are made for needed research into the interaction between task features, cognitive processing and language performance.

Item response theory and the assumption of unidimensionality for language tests

Considerable controversy has arisen around the assumption of unidimen sionality underlying the application of latent trait models of measurement. The intent of the present paper is to provide a clearer articulation of the unidimensionality assumption and to investigate the robustness and appli cability of a particular unidimensional model, the Rasch Model, for use with language proficiency tests that consist of batteries of subtests in a variety of skill areas and that are applied in the testing of the abilities of students from diverse educational, linguistic and cultural backgrounds. Results of the analysis of response data from the administration of a 150- item, five-subskill ESL proficiency/placement examination to 312 entering university students indicated that unidimensionality constraints were not violated.

Theoretical Perspectives on Reading

Reading text is a relatively recent human activity, only having been around for about 5,000 years. Through text, writers are able to communicate with others at great distances. Readers are able to perceive arbitrarily determined shapes presented against some background and then form them into meaning. As the reader becomes more proficient at reading and uses reading in more and different ways, this process of interpreting letters and words becomes increasingly automatic.

Trends in assessment scales and criterion-referenced language assessment

Two current developments reflecting a common concern in second/foreign language assessment are the development of: (1) scales for describing language proficiency/ability/performance; and (2) criterion-referenced performance assessments. Both developments are motivated by a perceived need to achieve communicatively transparent test results anchored in observable behaviors. Each of these developments in one way or another is an attempt to recognize the complexity of language in use, the complexity of assessing language ability, and the difficulty in interpreting potential interactions of scale task, trait, text, and ability. They reflect a current appetite for language assessment anchored in the world of functions and events, but also must address how the worlds of functions and events contain non skill-specific and discretely hierarchical variability. As examples of current tests that attempt to use performance criteria, the chapter reviews the Canadian Language Benchmark, the Common European Framework, and the Assessment of Language Performance projects.

A Criterion-Referenced Measurement Approach to ESL Achievement Testing.

The distinction between norm-referenced measurement (NRM) and criterion-referenced measurement (CRM) has become recognized as an issue in second-language testing. Traditional methods used to determine the reliability and validity of a test, as well as to analyse items for test improvement have been based on NRM principles. These traditional methods are not entirely appropriate for criterion-referenced tests designed to measure course achievement. This study presents approaches to test development item analysis, reliability and validity based on CRM principles. These CRM approaches are discussed and compared with NRM approaches in terms of the types of decisions which result from either approach. The study was conducted using data from an ESL achievement testing project currently in progress at UCLA. The results indicate that CRM approaches provide information not available through NRM methods.

Relationships among IRT item discrimination and item fit indices in criterion-referenced language testing:

This study investigates relationships among the IRT one-parameter fit statistics, the two-parameter slope parameter and traditional biserial correla tions in terms of the role these indices play in criterion-referenced language test construction. It discusses the assumptions of the two models and how these assumptions can affect criterion-referenced test construction and interpreta tion. The study then specifically examines how the indices interrelate as indices of item discrimination. Examinees in Mexico, Saudi Arabia and Japan were administered one of two forms of a functional test (Form A n = 430, k = 94: Form B n = 400, k = 95). The data were analysed using the two IRT models and the results were compared. The results indicate strong relationships among biserial correlation, two-parameter slope, and one-parameter infit and outfit. These results indicate the need to employ the two-parameter model when con ditions allow, and to take item discrimination and item difficulty indices into account when conditions do not. Further implications for interpreting the strong relationships between the indices are discussed.

Surrogate indices for item information functions in criterion-referenced language testing

Item analysis of criterion-referenced tests (CRTs) presents several practical problems. Traditional item discrimination indices may be of limited informa tiveness if score distributions are narrow. When no prior defined mastery group is available application of the CRT item difference index is not possible. In settings which have relatively small numbers of examinees, item-response theory (IRT) methods will not yield stable estimates. Like wise, in many language programs either IRT computer programs are unavailable or the results of IRT analysis will be uninformative to those involved in test development. This study examines the relationship of three item discrimination indices and the biserial correlation to IRT item informa tion functions (IIFs) in order to provide testers with information which will be useful in contexts in which IRT analysis is inappropriate. Three indices which indicate item discrimination at the cut-score are compared to IRT results on data from three types of language test data. The indices are the phi-coefficient (Φ), the B-index and the agreement statistic. The three types of language tests are 1) an ESL reading placement test, 2) an ESL reading achievement test and 3) an EFL multiple-choice reading cloze test. Implica tions and cautions for CRT development and analysis are presented.