Takashi Mitsuya

A cross-language study of compensation in response to real-time formant perturbation.

Past studies have shown that when formants are perturbed in real time, speakers spontaneously compensate for the perturbation by changing their formant frequencies in the opposite direction to the perturbation. Further, the pattern of these results suggests that the processing of auditory feedback error operates at a purely acoustic level. This hypothesis was tested by comparing the response of three language groups to real-time formant perturbations, (1) native English speakers producing an English vowel /ε/, (2) native Japanese speakers producing a Japanese vowel (/e([inverted perpendicular])/), and (3) native Japanese speakers learning English, producing /ε/. All three groups showed similar production patterns when F1 was decreased; however, when F1 was increased, the Japanese groups did not compensate as much as the native English speakers. Due to this asymmetry, the hypothesis that the compensatory production for formant perturbation operates at a purely acoustic level was rejected. Rather, some level of phonological processing influences the feedback processing behavior.

Language dependent vowel representation in speech production

The representation of speech goals was explored using an auditory feedback paradigm. When talkers produce vowels the formant structure of which is perturbed in real time, they compensate to preserve the intended goal. When vowel formants are shifted up or down in frequency, participants change the formant frequencies in the opposite direction to the feedback perturbation. In this experiment, the specificity of vowel representation was explored by examining the magnitude of vowel compensation when the second formant frequency of a vowel was perturbed for speakers of two different languages (English and French). Even though the target vowel was the same for both language groups, the pattern of compensation differed. French speakers compensated to smaller perturbations and made larger compensations overall. Moreover, French speakers modified the third formant in their vowels to strengthen the compensation even though the third formant was not perturbed. English speakers did not alter their third formant. Changes in the perceptual goodness ratings by the two groups of participants were consistent with the threshold to initiate vowel compensation in production. These results suggest that vowel goals not only specify the quality of the vowel but also the relationship of the vowel to the vowel space of the spoken language.

Auditory feedback and articulatory timing.

Talkers listen to their own voice while they speak and use that feedback to monitor and control fine details of speech production. When auditory feedback is perturbed in real time, talkers spontaneously alter their speech production to compensate for the perturbation. Most research using real‐time altered auditory feedback has focused on spectral manipulations of vowels with little attention devoted to temporal manipulations of consonants. In the present study, we examine the role of acoustic feedback in control of voice onset time (VOT). Utterances of the words “tip” and “dip” were recorded from native English speakers, and several representative productions were selected for each speaker. After this, talkers were asked to repeatedly produce either tip or dip. During these productions a real‐time processing system was used to provide modified feedback through headphones. When talkers said one word, they simultaneously heard their own voice saying the other word. Results showed that the speakers compensat...

Modulation of auditory-motor learning in response to formant perturbation as a function of delayed auditory feedback

The interaction of language production and perception has been substantiated by empirical studies where speakers compensate their speech articulation in response to the manipulated sound of their voice heard in real-time as auditory feedback. A recent study by Max and Maffett [(2015). Neurosci. Lett. 591, 25-29] reported an absence of compensation (i.e., auditory-motor learning) for frequency-shifted formants when auditory feedback was delayed by 100 ms. In the present study, the effect of auditory feedback delay was studied when only the first formant was manipulated while delaying auditory feedback systematically. In experiment 1, a small yet significant compensation was observed even with 100 ms of auditory delay unlike the past report. This result suggests that the tolerance of feedback delay depends on different types of auditory errors being processed. In experiment 2, it was revealed that the amount of formant compensation had an inverse linear relationship with the amount of auditory delay. One of the speculated mechanisms to account for these results is that as auditory delay increases, undelayed (and unperturbed) somatosensory feedback is given more preference for accuracy control of vowel formants.

A cross‐language study of compensatory response to formant‐shifted feedback.

Previous experiments in speech motor learning have demonstrated that the perception of our own voice while we speak plays a role in the control of fundamental and formant frequencies and vocal amplitude. When feedback is changed in real time, subjects alter speech production in attempt to compensate for the perturbation. By testing Japanese talkers in their native and a less familiar language (as well as English‐speaking controls), we examine how this perception‐production process is influenced by language. In the first study, native Japanese speakers produced an English word with formant‐shifted feedback. In the second experiment, native Japanese speakers produced a Japanese syllable with altered feedback and produced an English word that contained a similar vowel with normal feedback. The results were compared with data from English controls and suggest that the compensatory behavior is not language dependent.

Intentionality and categories in speech motor control

Actions are organized around goals or intentions. In speech production, there has been no agreement on how best to discuss speech goals. However, the auditory feedback perturbation methodology provides a window into the nature of speech goals. To the extent that subjects are sensitive to variation in an acoustic attribute, this attribute must be part of the controlled intention of articulation. In this presentation, we will review a series of studies that speak to this issue. In one study, we examined how intentionality of speech production influences compensatory formant production by instructing subjects to use a cognitive strategy in order to make the feedback sound consistent with the intended vowel. In other studies, we have explored the specificity of vowel formant compensation by comparing cross-language differences. The results indicate that speech goals are 1) very specific, defined by a phonemic category and its relationship with neighboring categories, and 2) multivariate. We will discuss these...

Stability of compensatory behavior for real-time perturbations of vowel formants

While we talk, not only do we listen to the speech sounds other people make but also we listen to our own voice in order to control the phonetic/phonological details of the sounds. One example of this relationship is demonstrated by real-time formant perturbation studies which show that talkers automatically change their formant production when auditory feedback does not match with the vowel they intend to produce. The reported results are consistent across studies, yet the variability between talkers is usually quite large, with some talkers showing a large magnitude compensation while others compensate modestly. To date, the degree to which talkers compensate has been assumed to be stable, but this has never been directly examined. The current study tested the stability of compensatory behavior for perturbed formant shifted feedback by repeatedly testing a group of talkers over the course of a few weeks to measure the variance in formant values and compensation across experimental sessions. The results ...