Paula Winke

Raters’ L2 background as a potential source of bias in rating oral performance

Based on evidence that listeners may favor certain foreign accents over others (Gass & Varonis, 1984; Major, Fitzmaurice, Bunta, & Balasubramanian, 2002; Tauroza & Luk, 1997) and that language-test raters may better comprehend and/or rate the speech of test takers whose native languages (L1s) are more familiar on some level (Carey, Mannell, & Dunn, 2011; Fayer & Krasinski, 1987; Scales, Wennerstrom, Richard, & Wu, 2006), we investigated whether accent familiarity (defined as having learned the test takers’ L1) leads to rater bias. We examined 107 raters’ ratings on 432 TOEFL iBTTM speech samples from 72 test takers. The raters of interest were L2 speakers of Spanish, Chinese, or Korean, while the test takers comprised three native-speaker groups (24 each) of Spanish, Chinese, and Korean. We analyzed the ratings using a multifaceted Rasch measurement approach. Results indicated that L2 Spanish raters were significantly more lenient with L1 Spanish test takers, as were L2 Chinese raters with L1 Chinese test takers. We conclude by concurring with Xi and Mollaun (2009, 2011) and Carey et al. that rater training should address raters’ linguistic background as a potential rater effect. Furthermore, we discuss the importance of recognizing rater L2 as a possible source of bias.

THE EFFECTS OF INPUT ENHANCEMENT ON GRAMMAR LEARNING AND COMPREHENSION

In his 2007 study “Effects of Textual Enhancement and Topic Familiarity on Korean EFL Students’ Reading Comprehension and Learning of Passive Form,” Lee demonstrated that learners were better able to correct written sentences that contained incorrect English passive forms after exposure to texts flooded with enhanced (versus nonenhanced) passive forms. But with enhanced forms, learners did worse on comprehension tests, which arguably demonstrated a trade-off: More attention to forms resulted in less to meaning. In this study, a conceptual replication of Lee’s using eye-movement data, I assessed how English passive construction enhancement affects English language learners’ (a) learning of the form (via pre- and posttest gains on passive construction tests) and (b) text comprehension. In contrast to Lee’s results, I found enhancement did not significantly increase form correction gain scores, nor did enhancement significantly detract from comprehension. There was no trade-off effect. Form learning and comprehension did not correlate. By recording learners’ eye movements while reading, I found enhancement significantly impacted learners’ noticing of the passive forms through longer gaze durations and rereading times. Thus, enhancement in this study functioned as intuitively and originally (Sharwood Smith, 1991 , 1993) proposed; it promoted noticing, but, in this case, without further explicit instruction, it appeared to have done little else.

The differences among three-, four-, and five-option-item formats in the context of a high-stakes English-language listening test

We adapted three practice College Scholastic Ability Tests (CSAT) of English listening, each with five-option items, to create four- and three-option versions by asking 73 Korean speakers or learners of English to eliminate the least plausible options in two rounds. Two hundred and sixty-four Korean high school English-language learners formed three groups. Each took three of the nine tests, one with five-option items, one with four-, and one with three-, with administrations counterbalanced to control for order and practice effects. Mean test scores of the three-option tests were significantly higher than those of four- and five-option tests. While no difference was found in mean item discriminations across the three different test formats, reliability coefficients showed inconsistent patterns depending on the number of options and test versions. One possible interpretation of the low correlations among the scores of three test formats is that items with different numbers of options tap into skills other than listening. The findings suggest that statistically, three options may or may not be optimal depending on the point of view taken – from that of the test score users, or from that of the test stakeholders. Test developers must consider multiple statistical, affective, and contextual factors in determining the optimal number of options.

Redefining the L2 Listening Construct Within an Integrated Writing Task: Considering the Impacts of Visual-Cue Interpretation and Note-Taking

Researchers debate whether listening tasks should be supported by visuals. Most empirical research in this area has been conducted on the effects of visual support on listening comprehension tasks employing multiple-choice questions. The present study seeks to expand this research by investigating the effects of video listening passages (vs. audio-only listening passages) on note-taking strategies and subsequent (integrated) writing task performance. Forty international students learning English at Michigan State University participated. Each wrote two English essays, 1 preceded by supporting video listening material and one by audio-only listening material with the order of presentation and the content of the listening material and writing task counter balanced (via Latin squares) to control for order and test-content effects. Participants completed an exit survey concerning their perceptions of the different tasks. Results indicate that there was no significant difference in performance between the two task types (video-based vs. audio-only-based); however, there was a significant difference in note-taking practices—as expected, when test takers watched the video, they took fewer notes. Qualitative data suggest that test takers had mixed perceptions of the usefulness of video. However, the majority of participants preferred the video-based task and believed that video helped them understand the listening passages.

THE PSYCHOLOGY OF THE LANGUAGE LEARNER: INDIVIDUAL DIFFERENCES IN SECOND LANGUAGE ACQUISITION

THE PSYCHOLOGY OF THE LANGUAGE LEARNER: INDIVIDUAL DIFFERENCES IN SECOND LANGUAGE ACQUISITION. Zoltan Dornyei . Mahwah, NJ: Erlbaum, 2005. Pp. 270.

Testing Hypotheses about Language Learning Using Structural Equation Modeling

69.95 cloth,

Training in research ethics among applied linguistics and SLA researchers

29.95 paper. This concise book on individual differences (IDs) in SLA presents research on topics such as language aptitude, motivation, cognitive styles, student self-regulation, and personality traits. Its in-depth treatment of the field will make it valuable to researchers in applied linguistics and second language (L2) studies. At the same time, its straightforward presentation and conversational tone should make it accessible to language teachers and students of L2 pedagogy interested in understanding more about what individual learners bring to the classroom.

Hot Potatoes version 5

In this chapter, I review a statistical method for hypothesis or theory testing called structural equation modeling (SEM). First, I describe what a model of second language acquisition (SLA) is. I do this so anyone, even those new to the field of applied linguistics, can understand the basic concepts underlying SEM; that is, SEM researchers first articulate a model of SLA, then get empirical data from the real world that operationalize the variables in the model. Researchers use an SEM program to test the model on the data (to see if the model fits the data; if the model is plausible in relation to the learning context of the people from whom the data were collected). After explaining the basics of SEM, I provide a review of 39 applied linguistics studies that have been published in the last five years (between 2008 and 2013) and that present at least one SEM analysis as part of the results. I discuss four problematic areas related to the use of SEM that I believe these 39 studies highlighted: (a) sample size, (b) model presentation, (c) reliability, and (d) the number of Likert-scale points. I conclude with possible solutions for the four problem areas and outline future directions.

Young learners’ response processes when taking computerized tasks for speaking assessment:

The paradigmatic case of ethical professional practice is the horkos commonly known as the Hippocratic Oath (Edelstein, 1943). Essentially, this involves invoking a curse on violators of commitments made in swearing an oath (or on those who swear insincerely). Originating in Ancient Greece, the Hippocratic Oath is associated with physicians swearing to practice medicine honestly and is the historical source of many formal procedures for ethical practice in western, and a variety of other, cultural traditions.

The feasibility of an automated eye-tracking-modified Fagan test of memory for human faces in younger Ugandan HIV-exposed children

Summary The Hot Potatoes program, which consists of modules for creating six different types of exercises, is an excellent resource for creating online, interactive language learning exercises that can be used in or out of the classroom. These types of exercises can be especially useful in language learning laboratories with internet access, or for remote learning. When matched with both appropriate content and motivated students, Hot Potatoes exercises seem likely to promote second language acquisition. Hot Potatoes user-friendly modules also allow more adept teachers to create complex, interactive exercises. Hot Potatoes is not intended as testing software, and should not be used as such. Nevertheless, the Hot Potatoes program suite provides teachers with flexible, easy-to-use modules for creating web-based language exercises that students can work on while also receiving feedback that will direct them towards correct answers.