Ann Pannekamp

Prosody-driven Sentence Processing: An Event-related Brain Potential Study

Four experiments systematically investigating the brains response to the perception of sentences containing differing amounts of linguistic information are presented. Spoken language generally provides various levels of information for the interpretation of the incoming speech stream. Here, we focus on the processing of prosodic phrasing, especially on its interplay with phonemic, semantic, and syntactic information. An event-related brain potential (ERP) paradigm was chosen to record the on-line responses to the processing of sentences containing major prosodic boundaries. For the perception of these prosodic boundaries, the so-called closure positive shift (CPS) has been manifested as a reliable and replicable ERP component. It has mainly been shown to correlate to major intonational phrasing in spoken language. However, to define this component as exclusively relying on the prosodic information in the speech stream, it is necessary to systematically reduce the linguistic content of the stimulus material. This was done by creating quasi-natural sentence material with decreasing semantic, syntactic, and phonemic information (i.e., jabberwocky sentences, in which all content words were replaced by meaningless words; pseudoword sentences, in which all function and all content words are replaced by meaningless words; and delexicalized sentences, hummed intonation contour of a sentence removing all segmental content). The finding that a CPS was identified in all sentence types in correlation to the perception of their major intonational boundaries clearly indicates that this effect is driven purely by prosody.

Sex hormone testosterone affects language organization in the infant brain.

Using a phonological discrimination paradigm, we show that the brain responses of 4-week-old infants systematically vary as a function of biological sex and testosterone level. Females who are generally low on testosterone demonstrated a clear phonological discrimination effect with a bilateral distribution. In male infants this effect systematically varied as a function of testosterone level. Males with high testosterone showed no discrimination effect, whereas males with low testosterone displayed a discrimination effect, which was clearly left-lateralized. The present data provide evidence for a strong influence of testosterone on language function and lateralization already present during the first weeks of life.

Prosodic processing at the sentence level in infants

We investigate whether 8-month-old infants can detect prosodic cues relevant in sentence structuring. We recorded event-related potentials to examine online responses to the processing of prosodic boundaries. Prior studies in adults have validated the closure positive shift as reflecting prosodic boundary perception during speech processing. The current study shows that in the event-related potentials of 8-month-olds, a closure positive shift is elicited in relation to the prosodic boundaries in speech, suggesting that these infants are able to structure speech input into prosodic units on a neurophysiological basis similar to that seen in adults. A delay in latency of the infant closure positive shift, however, suggests that childrens exploitation of prosodic boundaries for the segmentation of the speech stream is still developing.

Impact of phonological processing skills on written language acquisition in illiterate adults

Illiteracy remains a world-wide problem not only for children but also for adults. Phonological processing has been defined as a crucial factor for the acquisition of written language, which usually occurs in childhood. However, it is unclear to what extent phonological processing is necessary in order for adults to acquire written language skills. We tested 47 illiterate adults before and after a one-year alphabetization course in several cognitive domains relevant to phonological processing and compared their results to 41 matched controls who did not take part in the alphabetization course. Phonological awareness in the narrower sense (e.g., phoneme association) was a stronger predictor of alphabetization outcome than demographic variables such as years of education. In addition, despite improvement of illiterate individuals in phonological awareness, short-term memory, and visual attention from before to after the alphabetization course, they did not reach the phonological processing level of literate controls. Our results confirm that the alphabetization of adults requires and enhances phonological processes similar to those of children. Nevertheless, specific aspects, such as improvements in short-term memory or visual attention, need to be considered in order to improve and optimize alphabetization programs for adults.

In the eye of the listener: Pupil dilation elucidates discourse processing☆

The current study investigated cognitive resource allocation in discourse processing by means of pupil dilation and behavioral measures. Short question-answer dialogs were presented to listeners. Either the context question queried a new information focus in the successive answer, or else the context query was corrected in the answer sentence (correction information). The information foci contained in the answer sentences were either adequately highlighted by prosodic means or not. Participants had to judge the adequacy of the focus prosody with respect to the preceding context question. Prosodic judgment accuracy was higher in the conditions bearing adequate focus prosody than in the conditions with inadequate focus prosody. Latency to peak pupil dilation was longer when new information foci were perceived compared to correction foci. Moreover, for the peak dilation, an interaction of focus type and prosody was found. Post hoc statistical tests revealed that prosodically adequate correction focus positions were processed with smaller peak dilation in comparison to all other dialog conditions. Thus, pupil dilation and results of a principal component analysis suggest an interaction of focus type and focus prosody in discourse processing.

Present and past: Can writing abilities in school children be associated with their auditory discrimination capacities in infancy?

Literacy acquisition is highly associated with auditory processing abilities, such as auditory discrimination. The event-related potential Mismatch Response (MMR) is an indicator for cortical auditory discrimination abilities and it has been found to be reduced in individuals with reading and writing impairments and also in infants at risk for these impairments. The goal of the present study was to analyze the relationship between auditory speech discrimination in infancy and writing abilities at school age within subjects, and to determine when auditory speech discrimination differences, relevant for later writing abilities, start to develop. We analyzed the MMR registered in response to natural syllables in German children with and without writing problems at two points during development, that is, at school age and at infancy, namely at age 1 month and 5 months. We observed MMR related auditory discrimination differences between infants with and without later writing problems, starting to develop at age 5 months-an age when infants begin to establish language-specific phoneme representations. At school age, these children with and without writing problems also showed auditory discrimination differences, reflected in the MMR, confirming a relationship between writing and auditory speech processing skills. Thus, writing problems at school age are, at least, partly grounded in auditory discrimination problems developing already during the first months of life.

Facial speech gestures: The relation between visual speech processing, phonological awareness, and developmental dyslexia in 10-year-olds

Successful communication in everyday life crucially involves the processing of auditory and visual components of speech. Viewing our interlocutor and processing visual components of speech facilitates speech processing by triggering auditory processing. Auditory phoneme processing, analyzed by event-related brain potentials (ERP), has been shown to be associated with impairments in reading and spelling (i.e. developmental dyslexia), but visual aspects of phoneme processing have not been investigated in individuals with such deficits. The present study analyzed the passive visual Mismatch Response (vMMR) in school children with and without developmental dyslexia in response to video-recorded mouth movements pronouncing syllables silently. Our results reveal that both groups of children showed processing of visual speech stimuli, but with different scalp distribution. Children without developmental dyslexia showed a vMMR with typical posterior distribution. In contrast, children with developmental dyslexia showed a vMMR with anterior distribution, which was even more pronounced in children with severe phonological deficits and very low spelling abilities. As anterior scalp distributions are typically reported for auditory speech processing, the anterior vMMR of children with developmental dyslexia might suggest an attempt to anticipate potentially upcoming auditory speech information in order to support phonological processing, which has been shown to be deficient in children with developmental dyslexia.

Dissociating improvement of attention and intelligence during written language acquisition in adults

About one tenth of the world’s population cannot read and write sufficiently. Cognitive abilities, such as selective attention and crystallized as well as fluid intelligence, have been defined as crucial factors for the acquisition of written language skills. However, it is unclear whether these abilities are necessary also for the alphabetization of adults. Before and after a one-year alphabetization course, we compared the attention and intelligence of 47 illiterate individuals to 41 matched literate controls who did not take part in the alphabetization course. Illiterate individuals improved in selective attention and crystallized intelligence from before to after the alphabetization course; however, they did not reach the same level of functioning as literate controls. In addition, the fluid intelligence of illiterates did not improve. More importantly, when controlling for attention improvement, we found that improvement in crystallized intelligence was associated with alphabetization above and beyond the influence of attention. Our results suggest that alphabetization is closely related to improvements in attention and crystallized intelligence. Specifically, socio-cultural, knowledge- specific learning processes improve during the acquisition of written language skills and may not depend on only the enhancement of the ability to attend to relevant stimuli. Alphabetization programs may, therefore, benefit from distinct considerations of attentional, intellectual, and literacy related skill acquisitions.

Auditory phoneme discrimination in illiterates: Mismatch negativity – a question of literacy?

These days, illiteracy is still a major problem. There is empirical evidence that auditory phoneme discrimination is one of the factors contributing to written language acquisition. The current study investigated auditory phoneme discrimination in participants who did not acquire written language sufficiently. Auditory phoneme discrimination was analyzed in illiterate adults and literate controls by recording event-related brain potentials (ERPs) while participants listened to standard and deviant phonemes (oddball paradigm). The results showed that only literate controls yielded mismatch negativity (MMN), the characteristic ERP marker for auditory discrimination. In illiterates, no discernible MMN was observed. These findings indicate the importance of written language acquisition for the development and maintenance of auditory phoneme discrimination. The reduced ability in discriminating phonemes in adult illiterates suggests potential training measures concerning literacy acquisition in these adults.

Children with dyslexia show a reduced processing benefit from bimodal speech information compared to their typically developing peers

ABSTRACT During information processing, individuals benefit from bimodally presented input, as has been demonstrated for speech perception (i.e., printed letters and speech sounds) or the perception of emotional expressions (i.e., facial expression and voice tuning). While typically developing individuals show this bimodal benefit, school children with dyslexia do not. Currently, it is unknown whether the bimodal processing deficit in dyslexia also occurs for visual‐auditory speech processing that is independent of reading and spelling acquisition (i.e., no letter‐sound knowledge is required). Here, we tested school children with and without spelling problems on their bimodal perception of video‐recorded mouth movements pronouncing syllables. We analyzed the event‐related potential Mismatch Response (MMR) to visual‐auditory speech information and compared this response to the MMR to monomodal speech information (i.e., auditory‐only, visual‐only). We found a reduced MMR with later onset to visual‐auditory speech information in children with spelling problems compared to children without spelling problems. Moreover, when comparing bimodal and monomodal speech perception, we found that children without spelling problems showed significantly larger responses in the visual‐auditory experiment compared to the visual‐only response, whereas children with spelling problems did not. Our results suggest that children with dyslexia exhibit general difficulties in bimodal speech perception independently of letter‐speech sound knowledge, as apparent in altered bimodal speech perception and lacking benefit from bimodal information. This general deficit in children with dyslexia may underlie the previously reported reduced bimodal benefit for letter‐speech sound combinations and similar findings in emotion perception. HighlightsVisual‐auditory speech perception analyzed in children with and without dyslexia.Children with dyslexia showed reduced MMR for visual‐auditory speech perception.They also show reduced benefit from bimodal compared to monomodal speech.Early bimodal speech training for children at risk of dyslexia is recommended.