Neville Ryant

fMRI evidence for strategic decision-making during resolution of pronoun reference

Pronouns are extraordinarily common in daily language yet little is known about the neural mechanisms that support decisions about pronoun reference. We propose a large-scale neural network for resolving pronoun reference that consists of two components. First, a core language network in peri-Sylvian cortex supports syntactic and semantic resources for interpreting pronoun meaning in sentences. Second, a frontal-parietal network that supports strategic decision-making is recruited to support probabilistic and risk-related components of resolving a pronouns referent. In an fMRI study of healthy young adults, we observed activation of left inferior frontal and superior temporal cortex, consistent with a language network. We also observed activation of brain regions not associated with traditional language areas. By manipulating the context of the pronoun, we were able to demonstrate recruitment of dorsolateral prefrontal cortex during probabilistic evaluation of a pronouns reference, and orbital frontal activation when a pronoun must adopt a risky referent. Together, these findings are consistent with a two-component model for resolving a pronouns reference that includes neuroanatomic regions supporting core linguistic and decision-making mechanisms.

Highly accurate phonetic segmentation using boundary correction models and system fusion

Accurate phone-level segmentation of speech remains an important task for many subfields of speech research. We investigate techniques for boosting the accuracy of automatic phonetic segmentation based on HMM acoustic-phonetic models. In prior work [25] we were able to improve on state-of-the-art alignment accuracy by employing special phone boundary HMM models, trained on phonetically segmented training data, in conjunction with a simple boundary-time correction model. Here we present further improved results by using more powerful statistical models for boundary correction that are conditioned on phonetic context and duration features. Furthermore, we find that combining multiple acoustic front-ends gives additional gains in accuracy, and that conditioning the combiner on phonetic context and side information helps. Overall, we reduce segmentation errors on the TIMIT corpus by almost one half, from 93.9% to 96.8% boundary accuracy with a 20-ms tolerance.

Binding of Anaphors in LTAG

This paper presents an LTAG account for binding of reflexives and reciprocals in English. For these anaphors, a multi-component lexical entry is proposed, whose first component is a degenerate NP-tree that adjoins into the anaphors binder. This establishes the local structural relationship needed to ensure coreference and agreement. The analysis also allows a parallel treatment of reflexives and reciprocals, which is desirable because their behavior is very similar. In order to account for non-local binding phenomena, as in raising and ECM cases, we employ flexible composition, constrained by a subject intervention constraint between the two components of the anaphors lexical entry. Finally, the paper discusses further data such as extraction and picture-NP examples.

Automatic phonetic segmentation in Mandarin Chinese: Boundary models, glottal features and tone

We conducted experiments on forced alignment in Mandarin Chinese. A corpus of 7,849 utterances was created for the purpose of the study. Systems differing in their use of explicit phone boundary models, glottal features, and tone information were trained and evaluated on the corpus. Results showed that employing special one-state phone boundary HMM models significantly improved forced alignment accuracy, even when no manual phonetic segmentation was available for training. Spectral features extracted from glottal waveforms (by performing glottal inverse filtering from the speech waveforms) also improved forced alignment accuracy. Tone dependent models only slightly outperformed tone independent models. The best system achieved 93.1% agreement (of phone boundaries) within 20 ms compared to manual segmentation without boundary correction.

Mandarin tone classification without pitch tracking

A deep neural network (DNN) based classifier achieved 27.38% frame error rate (FER) and 15.62% segment error rate (SER) in recognizing five tonal categories in Mandarin Chinese broadcast news, based on 40 mel-frequency cepstral coefficients (MFCCs). The same architecture scored substantially lower when trained and tested with F0 and amplitude parameters alone: 40.05% FER and 22.66% SER. These results are substantially better than the best previously-reported results on broadcast-news tone classification [1] and are also better than a human listener achieved in categorizing test stimuli created by amplitude- and frequency-modulating complex tones to match the extracted F0 and amplitude parameters.

Automatic Analysis of Phonetic Speech Style Dimensions.

We apply automated analysis methods to create a multidimensional characterization of the prosodic characteristics of a large variety of speech datasets, with the goal of developing a general framework for comparing prosodic styles. Our datasets span styles including conversation, fluent reading, extemporized narratives, political speech, and advertisements; we compare several different languages including English, Spanish, and Chinese; and the features we extract are based on the joint distributions of F0 and amplitude values and sequences, speech and silence segment durations, syllable durations, and modulation spectra. Rather than focus on the acoustic correlates of a small number of discrete and mutually exclusive categories, we aim to characterize the space in which diverse speech styles live.

Large-scale analysis of Spanish /s/-lenition using audiobooks

Given forced alignment and accurate automatic phonetic classification and measurement, audiobooks are an important potential source of large-scale evidence about phonetic variation. For example, the audiobook version of the novel La Casa de los Espiritus, read by two Chilean actors, presents 17 hours of audio containing nearly 67,000 /s/ segments, distributed in a natural way across a wide variety of prosodic, lexical, morphological, syllabic, and phonetic environments. Thus we believe that this one audiobook offers more /s/ tokens than have been examined in the entire 50-year history of sociolinguistic study of Spanish /s/-lenition -- and analysis on this scale allows statistical evaluation of a much larger set of hypotheses about phonetic variation and its conditioning factors. For broad comparison of geographical variants, we can use audiobooks whose readers exhibit a variety of accent types, in this case comparing works read by Chilean, Argentinian, Mexican, and Peninsular speakers. Most of the sociolinguistic literature on variation in Spanish syllable-final /s/ treats it as involving three distinct categories: retained [s], aspirated [h], and deletion. In our data we see coherent gradient variation in the duration, frication strength, and laryngeal coarticulation of /s/, with aspiration, deletion and voicing as continuum endpoints.Given forced alignment and accurate automatic phonetic classification and measurement, audiobooks are an important potential source of large-scale evidence about phonetic variation. For example, the audiobook version of the novel La Casa de los Espiritus, read by two Chilean actors, presents 17 hours of audio containing nearly 67,000 /s/ segments, distributed in a natural way across a wide variety of prosodic, lexical, morphological, syllabic, and phonetic environments. Thus we believe that this one audiobook offers more /s/ tokens than have been examined in the entire 50-year history of sociolinguistic study of Spanish /s/-lenition -- and analysis on this scale allows statistical evaluation of a much larger set of hypotheses about phonetic variation and its conditioning factors. For broad comparison of geographical variants, we can use audiobooks whose readers exhibit a variety of accent types, in this case comparing works read by Chilean, Argentinian, Mexican, and Peninsular speakers. Most of the sociol...

Exploring Autism Spectrum Disorders Using HLT.

The phenotypic complexity of Autism Spectrum Disorder motivates the application of modern computational methods to large collections of observational data, both for improved clinical diagnosis and for better scientific understanding. We have begun to create a corpus of annotated language samples relevant to this research, and we plan to join with other researchers in pooling and publishing such resources on a large scale. The goal of this paper is to present some initial explorations to illustrate the opportunities that such datasets will afford.

Scale-space expansion of acoustic features improves speech event detection

In a system for detecting and measuring phonetic events (here bursts, voice onsets, and voice-onset times), we show that the addition of features smoothed at multiple scales can improve both recall (the proportion of events correctly identified) and measurement accuracy (the timing of events and the difference between event times, relative to expert human judgments). Multi-scale (or “scale space”) features had an especially strong positive effect on robustness across datasets with different materials and recording conditions. Standard machine-learning classifiers were able to integrate information across scales, without any special treatment of the multi-scale features.