Gary J. Ockey

The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students

The purpose of this study was to determine the extent to which performance on the TOEFL iBT speaking section is associated with other indicators of Japanese university students’ abilities to communicate orally in an academic English environment and to determine which components of oral ability for these tasks are best assessed by TOEFL iBT. To achieve this aim, TOEFL iBT speaking scores were compared to performances on a group oral discussion, picture and graph description, and prepared oral presentation tasks, and their component scores of pronunciation, fluency, grammar/vocabulary, interactional competence, descriptive skill, delivery skill, and question answering. Participants were Japanese university students (N = 222), who were English majors in a Japanese university. Pearson product–moment correlations, corrected for attenuation, between scores on the speaking section of TOEFL iBT and the three university tasks indicated strong relationships between the TOEFL iBT speaking scores and the three university tasks and high or moderate correlations between the TOEFL iBT speaking scores and the components of oral ability. For the components of oral ability, pronunciation, fluency, and vocabulary/grammar were highly associated with TOEFL iBT speaking scores while interactional competence, descriptive skill, and delivery skill were moderately associated with TOEFL iBT speaking scores. The findings suggest that TOEFL iBT speaking scores are good overall indicators of academic oral ability and that they are better measures of pronunciation, fluency and vocabulary/grammar than they are of interactional competence, descriptive skill, and presentation delivery skill.

Effects of Strength of Accent on an L2 Interactive Lecture Listening Comprehension Test

This article reports on a study which aimed to determine the effect of strength of accent on listening comprehension of interactive lectures. Test takers (N = 21,726) listened to an interactive lecture given by one of nine speakers and responded to six comprehension items. The test taker responses were analyzed with the Rasch computer program WINSTEPS to investigate the relative difficulty of the items associated with the nine versions of the interactive lectures. Results indicated that comprehension of interactive lectures was diminished with quite light accents, as has been found with monologic lectures.

Structural Equation Modeling Reporting Practices for Language Assessment

Studies that use structural equation modeling (SEM) techniques are increasingly encountered in the language assessment literature. This popularity has created the need for a set of guidelines that can indicate what should be included in a research report and make it possible for research consumers to judge the appropriateness of the interpretations made from a reported study. This article attempts to fill this void by providing a set of reporting guidelines appropriate for language assessment researchers.

The Structural Consistency of a Six-Factor Model of Academic Self-Concept Among Culturally Diverse Preadolescents in the United States

For decades, research has indicated that preadolescents’ self-concept is comprised of subject-specific academic factors, a general academic factor, and several nonacademic factors. More recently, there have been some indications that academic self-concept might further be differentiated into competence and affect factors, at least for some preadolescent ethnic groups. This study examines the extent to which a 6-factor model of academic self-concept, in which competence and affect are distinct, is tenable for White, Hispanic, and Native American preadolescents. A total of 313 White, 331 Hispanic, and 224 Native American preadolescents from the southwestern United States were administered a modified version of the SDQI. A confirmatory factor analysis (CFA) indicated that the 6-factor model of academic self-concept was tenable. A multiple-groups CFA suggested that the 6-factor model was largely group invariant between White and Hispanic and White and Native American preadolescents. However, important differences in the structures were identified.

Assessing L2 Listening

This book is relevant for language testers, listening researchers, and oral proficiency teachers, in that it explores four broad themes related to the assessment of L2 listening ability: the use of authentic, real-world spoken texts; the effects of different speech varieties of listening inputs; the use of audio-visual texts; and assessing listening as part of an interactive speaking/listening construct. Each theme is introduced with a review of the relevant literature, and then is examined through either two or three empirical studies. The notion of authenticity underlies each of these four themes. By creating more authentic test tasks that are similar to real world language tasks, test developers can create listening assessments that not only more effectively assess test takers’ communicative competence, but can also have a positive washback effect on educational systems.

Web-Based Virtual Environments for Facilitating Assessment of L2 Oral Communication Ability

ABSTRACT A growing number of stakeholders argue for the use of second language (L2) speaking assessments that measure the ability to orally communicate in real time. A Web-based virtual environment (VE) that allows live voice communication among individuals may have potential for aiding in delivering such assessments. While off-the-shelf voice communication VEs, such as Second Life, have been used for language learning and to a much lesser extent, language assessment, a very limited number of such environments have been designed with the explicit aim of aiding in assessing L2 oral communication. Moreover, test developers who desire to build such a VE have little theoretical basis to guide them. This conceptual article fills this void by discussing some of the theoretical issues that were considered when the researchers built a VE to help deliver L2 oral communication assessments.