Talia Isaacs

Disentangling accent from comprehensibility

The goal of this study was to determine which linguistic aspects of second language speech are related to accent and which to comprehensibility. To address this goal, 19 different speech measures in the oral productions of 40 native French speakers of English were examined in relation to accent and comprehensibility, as rated by 60 novice raters and three experienced teachers. Results showed that both constructs were associated with many speech measures, but that accent was uniquely related to aspects of phonology, including rhythm and segmental and syllable structure accuracy, while comprehensibility was chiefly linked to grammatical accuracy and lexical richness.

Deconstructing comprehensibility: identifying the linguistic influences on listeners' L2 comprehensibility ratings

Comprehensibility, a major concept in second language (L2) pronunciation research that denotes listeners’ perceptions of how easily they understand L2 speech, is central to interlocutors’ communicative success in real-world contexts. Although comprehensibility has been modeled in several L2 oral proficiency scales—for example, the Test of English as a Foreign Language (TOEFL) or the International English Language Testing System (IELTS)—shortcomings of existing scales (e.g., vague descriptors) reflect limited empirical evidence as to which linguistic aspects influence listeners’ judgments of L2 comprehensibility at different ability levels. To address this gap, a mixed-methods approach was used in the present study to gain a deeper understanding of the linguistic aspects underlying listeners’ L2 comprehensibility ratings. First, speech samples of 40 native French learners of English were analyzed using 19 quantitative speech measures, including segmental, suprasegmental, fluency, lexical, grammatical, and discourse-level variables. These measures were then correlated with 60 native English listeners’ scalar judgments of the speakers’ comprehensibility. Next, three English as a second language (ESL) teachers provided introspective reports on the linguistic aspects of speech that they attended to when judging L2 comprehensibility. Following data triangulation, five speech measures were identified that clearly distinguished between L2 learners at different comprehensibility levels. Lexical richness and fluency measures differentiated between low-level learners; grammatical and discourse-level measures differentiated between high-level learners; and word stress errors discriminated between learners of all levels.

Second language speech production: Investigating linguistic correlates of comprehensibility and accentedness for learners at different ability levels

The current project aimed to investigate the potentially different linguistic correlates of comprehensibility (i.e., ease of understanding) and accentedness (i.e., linguistic nativelikeness) in adult second language (L2) learners’ extemporaneous speech production. Timed picture descriptions from 120 beginner, intermediate, and advanced Japanese learners of English were analyzed using native speaker global judgments based on learners’ comprehensibility and accentedness, and then submitted to segmental, prosodic, temporal, lexical, and grammatical analyses. Results showed that comprehensibility was related to all linguistic domains, and accentedness was strongly tied with pronunciation (specifically segmentals) rather than lexical and grammatical domains. In particular, linguistic correlates of L2 comprehensibility and accentedness were found to vary by learners’ proficiency levels. In terms of comprehensibility, optimal rate of speech, appropriate and rich vocabulary use, and adequate and varied prosody were important for beginner to intermediate levels, whereas segmental accuracy, good prosody, and correct grammar featured strongly for intermediate to advanced levels. For accentedness, grammatical complexity was a feature of intermediate to high-level performance, whereas segmental and prosodic variables were essential to accentedness across all levels. These findings suggest that syllabi tailored to learners’ proficiency level (beginner, intermediate, or advanced) and learning goal (comprehensibility or nativelike accent) would be advantageous for the teaching of L2 speaking.

Phonological memory, attention control, and musical ability: Effects of individual differences on rater judgments of second language speech

This study examines how listener judgments of second language speech relate to individual differences in listeners’ phonological memory, attention control, and musical ability. Sixty native English listeners (30 music majors, 30 nonmusic majors) rated 40 nonnative speech samples for accentedness, comprehensibility, and fluency. The listeners were also assessed for phonological memory (serial recognition), attention control (trail making), and musical aptitude. Results showed that music majors assigned significantly lower scores than nonmusic majors solely for accentedness, particularly for low ability second language speakers. However, the ratings were not significantly affected by individual differences in listeners’ phonological memory and attention control, which implies that these factors do not bias listeners’ subjective judgments of speech. Implications for psycholinguistic research and for high-stakes speaking assessments are discussed.

Lexical profiles of comprehensible second language speech: the role of appropriateness, fluency, variation, sophistication, abstractness and sense relations

This study examined contributions of lexical factors to native-speaking raters’ assessments of comprehensibility (ease of understanding) of second language (L2) speech. Extemporaneous oral narratives elicited from 40 French speakers of L2 English were transcribed and evaluated for comprehensibility by 10 raters. Subsequently, the samples were analyzed for 12 lexical variables targeting diverse domains of lexical usage (appropriateness, fluency, variation, sophistication, abstractness, and sense relations). For beginner-to-intermediate speakers, comprehensibility was related to basic uses of L2 vocabulary (fluent and accurate use of concrete words). For intermediate-to-advanced speakers, comprehensibility was linked to sophisticated uses of L2 lexis (morphologically accurate use of complex, less familiar, polysemous words). These findings, which highlight complex associations between lexical variables and L2 comprehensibility, suggest that improving comprehensibility requires attention to multiple lexical domains of L2 performance.

On the supranational spell of PISA in policy

Abstract Background: PISA results appear to have a large impact upon government policy. The phenomenon is growing, with more countries taking part in PISA testing and politicians pointing to PISA results as reasons for their reforms. Purpose: The aims of this research were to depict the policy reactions to PISA across a number of jurisdictions, to see whether they exhibited similar patterns and whether the same reforms were evident. Sources of evidence: We investigated policy and media reactions to the 2009 and 2012 PISA results in six cases: Canada, China (Shanghai), England, France, Norway and Switzerland. Cases were selected to contrast high-performing jurisdictions (Canada, China) with average performers (England, France, Norway and Switzerland). Countries that had already been well reported on in the literature were excluded (Finland, Germany). Design and methods: Policy documents, media reports and academic articles in English, French, Mandarin and Norwegian relating to each of the cases were critically evaluated. Results: A policy reaction of ‘scandalisation’ was evident in four of the six cases; a technique used to motivate change. Five of the six cases showed ‘standards-based reforms’ and two had reforms in line with the ‘ideal-governance’ model. However, these are categorisations: the actual reforms had significant differences across countries. There are chronological problems with the notion that PISA results were causal with regard to policy in some instances. Countries with similar PISA results responded with different policies, reflecting their differing cultural and historical education system trajectories. Conclusions: The connection between PISA results and policy is not always obvious. The supranational spell of PISA in policy is in the way that PISA results are used as a magic wand in political rhetoric, as though they conjure particular policy choices. This serves as a distraction from the ideological basis for reforms. The same PISA results could motivate a range of different policy solutions.

Identifying Second Language Speech Tasks and Ability Levels for Successful Nurse Oral Interaction with Patients in a Linguistic Minority Setting: An Instrument Development Project

One of the most demanding situations for members of linguistic minorities is a conversation between a health professional and a patient, a situation that frequently arises for linguistic minority groups in North America, Europe, and elsewhere. The present study reports on the construction of an oral interaction scale for nurses serving linguistic minorities in their second language (L2). A mixed methods approach was used to identify and validate a set of speech activities relating to nurse interactions with patients and to derive the L2 ability required to carry out those tasks. The research included an extensive literature review, the development of an initial list of speech tasks, and validation of this list with a nurse focus group. The retained speech tasks were then developed into a questionnaire and administered to 133 Quebec nurses who assessed each speech task for difficulty in an L2 context. Results were submitted to Rasch analysis and calibrated with reference to the Canadian Language Benchmarks, and the constructs underlying the speech tasks were identified through exploratory and confirmatory factor analyses. Results showed that speech tasks dealing with emotional aspects of caregiving and conveying health-specific information were reported as being the most demanding in terms of L2 ability, and the most strongly associated with L2 ability required for nurse–patient interactions. Implications are discussed with respect to the development and use of assessment instruments to facilitate L2 workplace training for health care professionals.

Lexical correlates of comprehensibility versus accentedness in second language speech

The current project investigated the extent to which several lexical aspects of second language (L2) speech – appropriateness, fluency, variation, sophistication, abstractness, sense relations – interact to influence native speakers’ judgements of comprehensibility (ease of understanding) and accentedness (linguistic nativelikeness). Extemporaneous speech elicited from 40 French speakers of English with varied L2 proficiency levels was first evaluated by 10 native-speaking raters for comprehensibility and accentedness. Subsequently, the dataset was transcribed and analyzed for 12 lexical factors. Various lexical properties of L2 speech were found to be associated with L2 comprehensibility, and especially lexical accuracy (lemma appropriateness) and complexity (polysemy), indicating that these lexical variables are associated with successful L2 communication. In contrast, native speakers’ accent judgements seemed to be linked to surface-level details of lexical content (abstractness) and form (variation, morphological accuracy) rather than to its conceptual and contextual details (e.g., lemma appropriateness, polysemy).

Re-examining Phonological and Lexical Correlates of Second Language Comprehensibility: The Role of Rater Experience

Recently, some empirical studies (e.g., Isaacs & Trofimovich, 2012 in Studies in Second Language Acquisition, Cambridge University Press) have begun to examine in depth how native speakers attend to phonological (segmentals, syllables, word stress, intonation, speech rate) as well as lexical (frequency, diversity, polysemy, hypernymy, text length, lemma, morphology) aspects of language when evaluating comprehensibility (i.e., ease of understanding) in second language (L2) speech. The current study aimed to investigate whether and to what degree two types of native speaking raters (with and without professional experience in L2 assessment as trained linguists and ESL/EFL professionals) differentially assess L2 comprehensibility of extemporaneous speech samples produced by non-native speakers with a wide range of proficiency. Overall, the expert raters provided more lenient comprehensibility scores than the novice raters. The results of the linguistic analyses further revealed that while both of the expert and novice raters similarly processed phonological information (particularly drawing on prosody) during their comprehensibility judgement, they showed different processing patterns as to L2 lexical information. Unlike the novice raters’ comprehensibility judgement, which was exclusively linked with the number of different words L2 talkers used without much repetition in their L2 speech, the expert raters paid attention not only to surface details (diversity) but also to actual meanings (lemma) of L2 words. The results suggest that experienced raters’ leniency towards accented L2 speech may be related to their ability to assess L2 talkers’ appropriate use of words in context despite the non-nativelike phonological and lexical forms.Grants: European Commission: L2 COMMUNICATION - Fostering cross-cultural communication: Identifying the linguistic factors that promote comprehensibility in academic and workplace settings (303413) EUR 6,000 BPC fee funded by the EC FP7 Post-Grant Open Access Pilot

Flawed self-assessment: Investigating self- and other-perception of second language speech

This study targeted the relationship between self- and other-assessment of accentedness and comprehensibility in second language (L2) speech, extending prior social and cognitive research documenting weak or non-existing links between peoples self-assessment and objective measures of performance. Results of two experiments (N = 134) revealed mostly inaccurate self-assessment: speakers at the low end of the accentedness and comprehensibility scales overestimated their performance; speakers at the high end of each scale underestimated it. For both accent and comprehensibility, discrepancies in self- versus other-assessment were associated with listener-rated measures of phonological accuracy and temporal fluency but not with listener-rated measures of lexical appropriateness and richness, grammatical accuracy and complexity, or discourse structure. Findings suggest that inaccurate self-assessment is linked to the inherent complexity of L2 perception and production as cognitive skills and point to several ways of helping L2 speakers align or calibrate their self-assessment with their actual performance.