Emily Cibelli

The influence of lexical statistics on temporal lobe cortical dynamics during spoken word listening

Neural representations of words are thought to have a complex spatio-temporal cortical basis. It has been suggested that spoken word recognition is not a process of feed-forward computations from phonetic to lexical forms, but rather involves the online integration of bottom-up input with stored lexical knowledge. Using direct neural recordings from the temporal lobe, we examined cortical responses to words and pseudowords. We found that neural populations were not only sensitive to lexical status (real vs. pseudo), but also to cohort size (number of words matching the phonetic input at each time point) and cohort frequency (lexical frequency of those words). These lexical variables modulated neural activity from the posterior to anterior temporal lobe, and also dynamically as the stimuli unfolded on a millisecond time scale. Our findings indicate that word recognition is not purely modular, but relies on rapid and online integration of multiple sources of lexical knowledge.

The Sapir-Whorf Hypothesis and Probabilistic Inference: Evidence from the Domain of Color

The Sapir-Whorf hypothesis holds that our thoughts are shaped by our native language, and that speakers of different languages therefore think differently. This hypothesis is controversial in part because it appears to deny the possibility of a universal groundwork for human cognition, and in part because some findings taken to support it have not reliably replicated. We argue that considering this hypothesis through the lens of probabilistic inference has the potential to resolve both issues, at least with respect to certain prominent findings in the domain of color cognition. We explore a probabilistic model that is grounded in a presumed universal perceptual color space and in language-specific categories over that space. The model predicts that categories will most clearly affect color memory when perceptual information is uncertain. In line with earlier studies, we show that this model accounts for language-consistent biases in color reconstruction from memory in English speakers, modulated by uncertainty. We also show, to our knowledge for the first time, that such a model accounts for influential existing data on cross-language differences in color discrimination from memory, both within and across categories. We suggest that these ideas may help to clarify the debate over the Sapir-Whorf hypothesis.

The “Up” corpus: A corpus of speech samples across adulthood

Abstract We describe a speech corpus based on the “Up” series of documentary films by director Michael Apted, showing a set of individuals at seven year intervals over a period of 42 years. The corpus is meant to facilitate phonetic, psycholinguistic and sociolinguistic research on age-related change in speech during young and middle-age adulthood. The corpus contains audio files, transcripts time-aligned at the level of utterance, word, and segment, F0 and vowel formant measurements of portions of the films, featuring eleven participants at ages 21 through 49. The corpus is freely available to researchers upon request.

Sequence segmentation using joint RNN and structured prediction models

We describe and analyze a simple and effective algorithm for sequence segmentation applied to speech processing tasks. We propose a neural architecture that is composed of two modules trained jointly: a recurrent neural network (RNN) module and a structured prediction model. The RNN outputs are considered as feature functions to the structured model. The overall model is trained with a structured loss function which can be designed to the given segmentation task. We demonstrate the effectiveness of our method by applying it to two simple tasks commonly used in phonetic studies: word segmentation and voice onset time segmentation. Results suggest the proposed model is superior to previous methods, obtaining state-of-the-art results on the tested datasets.

Clinical correlates of aberrant conversational turn-taking in youth at clinical high-risk for psychosis

Studies on speech abnormalities in schizophrenia suggest that atypical turn-taking during conversation is a prominent feature of speech patterns in psychosis. These studies have typically reported inappropriate pause times based on pragmatic rating scales (e.g. Colle et al., 2013). However, these subjective assessments are susceptible to rater bias and provide only a broad perceptual analysis of vocal expression; specific, quantifiable speech parameters cannot be assessed. While impairments in turn-taking are pervasive in formal psychosis samples, it is unclear whether these anomalies exist in individuals at clinical high risk (CHR) for psychotic disorders. Because language data is easy to collect and conversation is a typical part of clinical assessments, linguistic patterns such as atypical turn-taking may be a readily-accessible domain to search for early biomarkers for psychosis risk. These patterns may also improve etiological understanding (highlighting foundational contributors to social and functional deficits) and identify novel treatment targets in the psychosis prodrome. The present study evaluated turn-taking from speech data provided by CHR youth and healthy controls to address two goals: 1) evaluate if CHR individuals exhibit abnormalities in turn-taking and 2) investigate if turn-taking performance is correlated with symptom severity within the CHR group. Our datawas taken from clinical interviewswith the duration of between-turn pauses (BTPs) preceding responses as our metric. Based on findings in schizophrenia, we predicted that CHR youth would exhibit deficits in turn-taking performance. Further, consistent with Colle et al. (2013), who found a correlation between stronger clinical symptoms and poorer pragmatic performance in patients with schizophrenia, we predicted that turn-taking abnormalities would correlate with elevated prodromal symptoms. Following Cohen et al. (2016), who suggest that vocal parameters in speech of patients with schizophrenia are contextand demography-dependent, our analyses controlled for age and sex. Based on previous studies of turn-taking in the general population, we controlled for the complexity and duration of the participants response and the duration of the question preceding the BTP (Barthel et al., 2015; Casillas et al., 2015). Finally, we controlled for context (Cohen et al., 2016) by contrasting BTP durations following informal background questions (about daily habits, family, etc.) and structured interview questions. Data was taken from structured clinical interviews with 34 neuroleptic-free CHR youth and 36 age-matched healthy controls. Participants were administered the Structural Interview for Prodromal Syndromes (SIPS; Miller et al., 2003), the Prodromal Inventory of Negative Symptoms (Pelletier-Baldelli et al., 2017), the Structured Clinical Interview for DSM-IV (SCID-I; First et al., 2015), the Global Functioning

The influence of lexical selection disruptions on articulation.

Interactive models of language production predict that it should be possible to observe long-distance interactions; effects that arise at one level of processing influence multiple subsequent stages of representation and processing. We examine the hypothesis that disruptions arising in nonform-based levels of planning—specifically, lexical selection—should modulate articulatory processing. A novel automatic phonetic analysis method was used to examine productions in a paradigm yielding both general disruptions to formulation processes and, more specifically, overt errors during lexical selection. This analysis method allowed us to examine articulatory disruptions at multiple levels of analysis, from whole words to individual segments. Baseline performance by young adults was contrasted with young speakers’ performance under time pressure (which previous work has argued increases interaction between planning and articulation) and performance by older adults (who may have difficulties inhibiting nontarget representations, leading to heightened interactive effects). The results revealed the presence of interactive effects. Our new analysis techniques revealed these effects were strongest in initial portions of responses, suggesting that speech is initiated as soon as the first segment has been planned. Interactive effects did not increase under response pressure, suggesting interaction between planning and articulation is relatively fixed. Unexpectedly, lexical selection disruptions appeared to yield some degree of facilitation in articulatory processing (possibly reflecting semantic facilitation of target retrieval) and older adults showed weaker, not stronger interactive effects (possibly reflecting weakened connections between lexical and form-level representations).

Prosodic cues to psychosis risk

Schizophrenia is known to impact prosody, often described as “flat affect.” Individuals with schizophrenia show reduced F0 variability relative to neurotypical individuals (Rapcan et al. 2010). However, the speech of adolescents at high risk for psychosis in the prodromal (pre-diagnosis) stage has not been investigated, leaving open the question of whether speech prosody might be an early signal of symptoms associated with psychotic disorders. To investigate this issue, the speech of 18 ultra high-risk (UHR) youth (ages 16-21) was compared to 18 age- and gender-matched controls. F0 (pre-processed for smoothing and error correction) was extracted from 10-minute segments of speech recorded during clinical interviews. Using LDA classification, F0 summary statistics (mean and variability) separated male UHR and control speakers (69% accuracy) but not female speakers (42% accuracy), consistent with gender differences in psychosis onset during adolescence (Ochoa et al., 2012). Linear models of symptoms measured...

The role of articulatory cues in the establishment of perceptual categories

This study investigates the interaction of perceptual and articulatory information in the acquisition of non-native phoneme categories. There is evidence that perceptual learning can improve production of target sounds in a new language, but the reverse—improvement of perceptual discrimination from articulatory learning—is less well-studied (for one example, see Catford and Pisoni 1970). In this experiment, native English speakers learned Hindi coronal stop contrasts (dental vs. retroflex; 4-way VOT contrast) in a pre-test/training/post-test paradigm. Training involved production training, where subjects got explicit instruction about place of articulation and voicing of the target sounds, and practiced producing them. Comparison of pre- and post-test performance on an AX discrimination task indicated greater accuracy (β = 0.486, t = 2.763) and faster responses (β = −0.215, t = −3.396) for most target contrasts after training. (Those contrasts which did not improve tended to map onto English phonemic distinctions, and were well-discriminated at pre-test.) This result supports the hypothesis that articulatory information can contribute to the early development of novel perceptual categories under certain conditions. More generally, it suggests that information from one speech domain can be used to support representations in another speech domain.

Multiple sources of information contribute to novel category formation

The acquisition of novel phonemes in a new language often presents a challenge for learners, particularly when target categories overlap with or assimilate to native categories. In the current study, English speakers who have no experience with Hindi are asked to learn the Hindi dental-retroflex place contrast and the four-way stop voicing contrast. The multi-day study includes an AX discrimination task and a repetition task with (V)CV syllables. In experiment 1, a training procedure was designed to manipulate multiple sources of information available to the listener. Training sessions include performance feedback and adaptive fading (a progression in exposure from clear tokens to more peripheral exemplars). Critically, the study also includes explicit articulatory training of the target sounds, to test the hypothesis that information about the existence of a new articulatory target can support the development of a perceptual category. Experiment 2, a control study, tests whether simple exposure to stimul...

Perceptual compensation with familiar and unfamiliar rounding coarticulation

We compared the integration of three kinds of contextual information in the perception of the fricatives [s] and [∫]. We asked American English listeners to identify sounds on an [s] to [∫] continuum and manipulated (1) the vowel context of the fricative ([Ce], [Co], [Cœ]), (2) the original fricative of the CV ([s] vs [∫]), and (3) the modality of the stimulus (audio-only, AV). There was a large compensation for coarticulation effect on perception—subjects responded with “s” more often when the following vowel was round. Interestingly, and perhaps significantly, perceptual compensation was not as great with the less familiar vowel [œ] even when listeners saw the face. Measurements of lip rounding in these stimuli show that [o] and [œ] have about the same degree and type of rounding over the CV. In a second experiment, we measured reaction time to audio-visual mismatches in these stimuli (again in a fricative identification task). We found that mismatches of audio and video consonant information slowed rea...