Sheila R. Brutten

First and Second Language Reading Comprehension.

Random parallel reading comprehension tests in Japanese and English were administered to a sample of native Japanese students enrolled in intensive English instruction at three different levels of English language proficiency as assessed by an independent measure. Evidence for a threshold competence ceiling at which first language reading abilities transferred to second language reading abilities was found. At the highest proficiency level, those readers who scored high on the first language reading test also systematically scored high on the second language reading test. Pedagogical implications of the study are discussed.

From Conventional to Computer-Adaptive Testing of ESL Reading Comprehension.

Abstract In this paper we describe the development of a computer-adaptive test of ESL reading comprehension. The computer-adaptive test was based on items from a battery of four conventional fixed-form tests and was constructed using an adaptive test development system for the Macintosh HyperCAT. The application of computer-adaptive testing (CAT) to the assessment of reading comprehension and other abilities in a second language serves as an illustration in this paper. We discuss the constraints that apply to CAT and the substantial advantages that CAT has over conventional testing modalities. The constraints on CAT include: the unidimensionality of the test, the homogeneity of the test population, and a neutral test method effect.

An investigation of patterns of discontinuous learning: implications for ESL measurement

Research from cognitive psychology, L2 acquisition and psychometry sug gests that competence/proficiency is multidimensional, and that proficiency does not always develop at the same rate in all its domains. In particular, L2 acquisition research indicates that competence does not always develop in a linear, monotonic pattern at an equal rate in all its component parts owing to restructuring and reorganization. Rehearsal, automatic and controlled pro cessing, creative simplification, task constraints and attention to form have been adduced to explain restructuring and reorganization. Classical test theory and item response theory models are not consonant with findings from cognitive psychology and L2 acquisition, because these models pinpoint individuals on a single continuum of proficiency. The study reported here tested the experimental hypothesis that there would not be a monotonic relationship between increasing competence in different linguistic domains and growth in L2 proficiency. There was general support for the hypothesis. The article concludes with a brief introduction to some assessment models which can accommodate discontinuous patterns of growth. The models examined can be used to describe and model a students qualitative cognitive state and knowledge structures.

A comparative item analysis study of a language testing instrument

The goals of this study were threefold: (1) to provide a comparative analysis of five different item analysis indices using both IRT and non-IRT indices; (2) to describe the characteristics of flagged items; and (3) to investigate the appropriateness of logistic regression as an item analysis technique for further studies. The performance of five item analysis indices was examined in this study; p-value, point-biserial correlation, Rasch b parameter, Rasch χ2 fit and χ2 from the logistic regression model. A factor analysis indicated that the first factor was a difficulty factor and included the Rasch b and p-value indices, whereas the second factor represented the discrimination power of an item and consisted of the Rasch χ2 fit, χ2 from the logistic regression and the point-biserial correlation. There was a tendency for difficult items to exhibit more unusual response patterns. The χ 2 from the logistic regression fit was found to be an appropriate index when a researcher is attempting to detect items that function differentially and when the subject pool consists of varying ability groups.

An Analysis of Student and Teacher Indications of Vocabulary Difficulty

years, glossing has been seen as a necessary procedure because reading for meaning depends, in part, on an understanding of the words that comprise a passage. Comprehension of meaning can be seriously impaired by the presence of a limited vocabulary (Chastain 1976, Cowan 1974, Eskey 1971). This is not to say that comprehension depends on an understanding of every word in a passage (Been 1975). Content can be understood and reading can be facilitated if the words that are important for meaning are known. This is evidenced by the fact that fluent readers pass over the occasional words or phrases that are not within their sight vocabularies and still comprehend the overall meaning of the passage (Smith 1973).

A Factor Analysis of Direct and Indirect Measures of English as a Second Language Writing Ability

This paper reports the results of a study to determine whether an analysis of objective test measures and holistically scored composition data would support a single or a multifactor theory of writing assessment and to determine which category of measure-direct or indirect-was the more reliable. Four experimental objective tests of writing ability based on an information-processing approach were administered to one hundred foreign university students for whom English was not their native language. The subjects also wrote descriptive essays which were holistically graded. The results indicated that the holistic and objective assessments were relatively independent measures of writing ability, indicating support for a two-factor theory of ability assessment. The holistic assessment was found to be more reliable than the objective assessment.

A Behavioral Anchoring Analysis of Three ESL Reading Comprehension Tests.

This article reports the results of a behavioral anchoring analysis of three ESL reading comprehension tests. For each test, anchor points on a continuum of ability level were selected for analysis. Items that discriminated between adjacent anchor points were batched and analyzed in terms of their relation to the structure of the text, the readers prior knowledge, and the cognitive processes required to answer the question. The results indicated that for each of three proficiency levels (Levels 2, 3, and 4), the higher ability students could comprehend micropropositions, whereas the lower ability students could not. The higher ability students at Levels 3 and 4 could comprehend questions whose sources of information were implicit, whereas the lower ability students at these levels could not. Higher ability students and lower ability students at all proficiency levels showed competence with linguistic structures that related parts of the text. And finally, the derivational complexity and the readability levels of the texts covaried with an increase in proficiency level.

An Item Discriminability Study of Textually Explicit, Textually Implicit, and Scriptally Implicit Questions

This paper reports the effect that background knowledge has on the item discriminability of reading comprehension items; item discriminability is an index of how well a test item discriminates bet ween weak and strong examinees in the ability being tested. The questions from three reading comprehension tests were placed into one of the following categories: (1) textually explicit; (2) textually implicit; and (3) scriptally implicit. An item discriminability index was computed for each item. The results indicated that there were significant differences in the three categories of itèms with respect to item discriminability and that reading comprehension items which are heavily dependent on background knowledge do not ex hibit good item discriminability. The findings indicate that researchers must control for background knowledge in a reading test when they use test products as the basis for making inferences about the pro cesses underlying reading comprehension.

Using Cognitive Grammar, Natural Phonology, and Item Response Theory to Explain ESL Subjects’ Control of the [D] and [Z] Morphemes

Our purpose in this paper is to apply selected notions from cognitive grammar, natural phonology, and item response theory in order to give better insight on ESL subjects’ processing of the D and Z morphemes in English.

Word Frequency and Contextual Richness in Esl Word Identification

There has been considerable research conducted on the effects of meaning, frequency, context, and redundancy on word identification in first language (hereafter L1) reading comprehension. The first part of this paper will review some of the major research that has been presented concerning each of these factors.