László Hunyadi

ERP correlates of prosody and syntax interaction in case of embedded sentences

Abstract Understanding spoken language depends on processing the delicate combination of grammatical structure, meaning and prosody of utterances. Previous studies have established that prosody influences the processing of sentences when the grammatical structure is ambiguous, however it is unclear how closely prosody and syntax are related when there is no ambiguity. In an event-related brain potential (ERP) study, we investigated the processing of embedded normal and pseudosentences in which all function and content words were replaced by meaningless words. Sentences could have either natural prosodic structure or incongruent prosodic structure, where the prosody deviated from the one expected based on the syntactic structure, but otherwise the sentences were unambiguous. The resulting ERP components (CPS) showed that the construction of prosodic structure was similar in normal and pseudosentences, thus suggesting that prosody has an abstract, recursive representation, independent of other linguistic information. Moreover, we found evidence that the incongruent prosody was not only detected (shown by the RAN), but it induced neural reintegration processes (shown by the P600) in spite of the syntactic structure of sentences being intact. These results suggest that the prosodic structure is a mandatory constituent of sentence structure building whenever it is present.

The effects of attention and task-relevance on the processing of syntactic violations during listening to two concurrent speech streams

The notion of automatic syntactic analysis received support from some event-related potential (ERP) studies. However, none of these studies tested syntax processing in the presence of a concurrent speech stream. Here we present two concurrent continuous speech streams, manipulating two variables potentially affecting speech processing in a fully crossed design: attention (focused vs. divided) and task (lexical – detecting numerals vs. syntactical – detecting syntactic violations). ERPs elicited by syntactic violations and numerals as targets were compared with those for distractors (task-relevant events in the unattended speech stream) and attended and unattended task-irrelevant events. As was expected, only target numerals elicited the N2b and P3 components. The amplitudes of these components did not significantly differ between focused and divided attention. Both task-relevant and task-irrelevant syntactic violations elicited the N400 ERP component within the attended but not in the unattended speech stream. P600 was only elicited by target syntactic violations. These results provide no support for the notion of automatic syntactic analysis. Rather, it appears that task-relevance is a prerequisite of P600 elicitation, implying that in-depth syntactic analysis occurs only for attended speech under everyday listening situations.

Incompleteness and Fragmentation: Possible Formal Cues to Cognitive Processes Behind Spoken Utterances

What may eventually connect engineers and linguists most is their common interest in language, more specifically language technology: engineers build more and more intelligent robots desirably communicating with humans through language. Linguists wish to verify their theoretical understanding of language and speech through practical implementations. Robotics is then a place for the two to meet. However, speech, especially within spontaneous communication seems to often withstand usual generalizations: the sounds you hear are not the sounds you describe in a laboratory, the words you read in a written text may be hard to identify by speech segmentation, the sequences of words that make up a sentence are often too fragmented to be considered a “real” sentence from a grammar book. Yet, humans communicate, and this is most often, successful. Typically this is achieved through cognition, where people not only use words, these are used in context. People also use words in semantic context, by combining voices and gestures , in a dynamically changing, multimodal situational context. Each individual does not simply pick out words from the flow of a verbal interaction, but also observes and reacts to other, using multimodal cues as a point of reference and inference making navigation in communication. It is reasonable to believe that participants in a multimodal communication event follow a set of general, partly innate rules based on a general model of communication. The model presented below interperate numerous forms of dialogue by uncovering their syntax , prosody and overall multimodality within the HuComTech corpus of Hungarian. The research aims at improving the robustness of the spoken form of natural language technology.