Jyrki Tuomainen

The combined perception of emotion from voice and face: early interaction revealed by human electric brain responses

Judgement of the emotional tone of a spoken utterance is influenced by a simultaneously presented face expression. The time course of this integration was investigated by measuring the mismatch negativity (MMN). In one condition, the standard stimulus was an angry voice fragment combined with a (congruous) angry face expression. In the deviant pair, the voice expression was kept the same and only the face expression changed to an (incongruous) sad face. The pairs with a deviant visual item evoked a negative electric brain response showing the characteristics of the MMN, which is usually evoked only by auditory deviations. Similar results were obtained by employing incongruous standard and congruous deviant pairs. These findings provide compelling evidence of an early integration of face with voice information in the processing of affect.

Cortical Differences in Tonal versus Vowel Processing as Revealed by an ERP Component Called Mismatch Negativity (MMN)

Event-related potentials were recorded from four aphasic subjects in order to study if discrimination of synthetic vowels is impaired by left posterior brain damage. A component called the mismatch negativity (MMN) which is assumed to reflect basic discriminatory processes of auditory stimuli was measured. In accordance with the hypothesis, two patients with posterior lesions failed to show any MMN response to synthetic vowels, whereas two patients with predominantly anterior lesions produced the response. The fact that all four patients produced an MMN response to sine wave stimuli indicates that the result does not reflect an across-the-board effect.

Audio-visual speech perception is special

In face-to-face conversation speech is perceived by ear and eye. We studied the prerequisites of audio-visual speech perception by using perceptually ambiguous sine wave replicas of natural speech as auditory stimuli. When the subjects were not aware that the auditory stimuli were speech, they showed only negligible integration of auditory and visual stimuli. When the same subjects learned to perceive the same auditory stimuli as speech, they integrated the auditory and visual stimuli in a similar manner as natural speech. These results demonstrate the existence of a multisensory speech-specific mode of perception.

Native and foreign vowel discrimination as indexed by the mismatch negativity (MMN) response

The development of a new vowel category was studied by measuring both automatic mismatch negativity and conscious behavioural target discrimination. Three groups, nai;ve Finns, advanced Finnish students of English, and native speakers of English, were presented with one pair of Finnish and three pairs of English synthetic vowels. The aim was to determine whether the advanced student group would show native-like responses to the unfamiliar vowel contrasts of the target language. The results suggest that learning in classroom environment may not lead to the formation of new long-term native-like memory traces.

Recognition of morphologically complex words in Finnish: Evidence from event-related potentials

The temporal dynamics of processing morphologically complex words was investigated by recording event-related brain potentials (ERPs) when native Finnish-speakers performed a visual lexical decision task. Behaviorally, there is evidence that recognition of inflected nouns elicits a processing cost (i.e., longer reaction times and higher error rates) in comparison to matched monomorphemic words. We aimed to reveal whether the processing cost stems from decomposition at the early visual word form level or from re-composition at the later semantic-syntactic level. The ERPs showed no early effects for morphology, but revealed an interaction with word frequency at a late N400-type component, as well as a late positive component that was larger for inflected words. These results suggest that the processing cost stems mainly from the semantic-syntactic level. We also studied the features of the morphological decomposition route by investigating the recognition of pseudowords carrying real morphemes. The results showed no differences between inflected vs. uninflected pseudowords with a false stem, but differences in relation to those with a real stem, suggesting that a recognizable stem is needed to initiate the decomposition route.

Automatic discrimination of phonetically relevant and irrelevant vowel parameters as reflected by mismatch negativity

An auditory event-related brain potential called mismatch negativity (MMN) was measured to study the perception of vowel pitch and formant frequency. In the MMN paradigm, deviant vowels differed from the standards either in F0 or F2 with equal relative steps. Pure tones of corresponding frequencies were used as control stimuli. The results indicate that the changes in F0 or F2 of vowels significantly affected the MMN amplitudes. The only variable significantly affecting the MMN latencies was sex which, however, did not have any effect on the amplitudes of the MMN. As expected, the MMN amplitudes increased with an increase in the acoustical difference between the standards and the deviants in all cases. On the average, the amplitudes were lower for the vowels than for the pure tones of equal loudness. However, in vowels, minor frequency changes in F0 produced higher MMN amplitudes than similar relative changes in F2. It was also noted that even the smallest and phonetically irrelevant change in F2 was detected by the MMN process. In overall, the results demonstrate that the MMN can be measured separately for F0 and F2 of vowels, although the MMN responses show large interindividual differences.

Adaptation of the Boston diagnostic aphasia examination and the boston naming test into finnish

We have adapted two widely used aphasia tests, the Boston Diagnostic Aphasia Examination (BDAE) and the Boston Naming Test (BNT), to the Finnish language. The BDAE is the first extensive standardized aphasia test battery that will be published in Finnish. The test adaptations as well as reliability and validity information of the Finnish BDAE version are described. The Finnish versions of both BDAE and BNT are expected to gain widespread use among speech therapists and neuropsychologists working with Finnish-speaking subjects.

The mismatch negativity and reaction time as indices of the perceptual distance between the corresponding vowels of two related languages.

Two experiments were conducted to determine whether vowel familiarity affects automatic and conscious vowel discrimination. Familiar (Finnish) and unfamiliar (Komi) vowels were presented to Finnish subjects. The good representatives of Finnish and Komi mid vowels were grouped into three pairs: front /e- epsilon /, central /ø-oe/, and back /o-o/. The acoustic difference for /e- epsilon / and /o-o/ was smaller than that for /ø-oe/. For /e- epsilon /, the Komi vowel / epsilon / was at the boundary between the Finnish /e/ and /ae/. The stimuli were presented in an oddball paradigm. In three different blocks, each Komi vowel in turn served as the standard (probability 0.86) and the corresponding Finnish vowel as the deviant stimulus (probability 0.14), and vice versa. In Experiment 1, subjects were instructed to press a button as soon as they detected a deviant stimulus. In Experiment 2, event-related potentials (ERPs) were recorded to these stimuli in order to use the mismatch negativity (MMN) as an index of the perceptual distance between the members of each vowel pair, while subjects did not attend to the stimuli. There were similar effects of the acoustic distance within a vowel pair for both the reaction time (RT) and the MMN amplitude; the RT decreased and the MMN amplitude increased when the acoustic difference between the stimuli increased. However, the RT was longer when the Komi / epsilon / was the standard and the Finnish /e/ was the deviant than vice versa. No such pattern was found for the MMN. Thus, the phonemic status of the standard stimulus seems to play a role at the attentive but not at the pre-attentive level.

Multiple oral rereading technique in rehabilitation of pure alexia

Abstract Three pure alexic patients were given reading practice with the multiple oral re-reading (MOR) technique (Moyer 1979). All patients read single words relatively fast, but differed from each other in the reading speed of texts. In addition, two of the patients (HT and TT) had no significant memory or visuospatial deficit whereas one patient (PA) exhibited severe memory and visuospatial problems. The results showed in HT and TT a notable increase in reading speed of texts after therapy when compared to baseline measures. In relationship to text reading, single word reading became significantly faster only in patient HT. Patient PA did not benefit from MOR. The difference in the recovery of word reading versus text reading suggests that MOR does not primarily affect the functioning of the damaged word-form system. Instead, the increase in reading speed may be attributed to top-down processes, which facilitate the recognition of words in a sentence frame. Furthermore, it is suggested that this kind o...

Multistage audiovisual integration of speech: dissociating identification and detection

Speech perception integrates auditory and visual information. This is evidenced by the McGurk illusion where seeing the talking face influences the auditory phonetic percept and by the audiovisual detection advantage where seeing the talking face influences the detectability of the acoustic speech signal. Here, we show that identification of phonetic content and detection can be dissociated as speech-specific and non-specific audiovisual integration effects. To this end, we employed synthetically modified stimuli, sine wave speech (SWS), which is an impoverished speech signal that only observers informed of its speech-like nature recognize as speech. While the McGurk illusion only occurred for informed observers, the audiovisual detection advantage occurred for naïve observers as well. This finding supports a multistage account of audiovisual integration of speech in which the many attributes of the audiovisual speech signal are integrated by separate integration processes.