Ina Bornkessel-Schlesewsky

The role of prominence information in the real time comprehension of transitive constructions: A cross-linguistic approach

Approaches to language processing have traditionally been formulated with reference to general cognitive concepts (e.g. working memory limitations) or have based their representational assumptions on concepts from generative linguistic theory (e.g. structure determines interpretation). Thus, many well-established generalisations about language that have emerged from cross-linguistic/typological research have not as yet had a major influence in shaping ideas about online processing. Here, we examine the viability of using typologically motivated concepts to account for phenomena in online language comprehension. In particular, we focus on the comprehension of simple transitive sentences (i.e. sentences involving two arguments/event participants) and cross-linguistic similarities and differences in how they are processed. We argue that incremental argument interpretation in these structures is best explained with reference to a range of cross-linguistically motivated, hierarchically ordered information types termed ‘prominence scales’ (e.g. animacy, definiteness/specificity, case marking and linear order). We show that the assumption of prominence-based argument processing can capture a wide range of recent neurocognitive findings, as well as deriving well-known behavioural results.

Reconciling time, space and function: A new dorsal-ventral stream model of sentence comprehension

We present a new dorsal-ventral stream framework for language comprehension which unifies basic neurobiological assumptions (Rauschecker & Scott, 2009) with a cross-linguistic neurocognitive sentence comprehension model (eADM; Bornkessel & Schlesewsky, 2006). The dissociation between (time-dependent) syntactic structure-building and (time-independent) sentence interpretation assumed within the eADM provides a basis for the division of labour between the dorsal and ventral streams in comprehension. We posit that the ventral stream performs time-independent unifications of conceptual schemata, serving to create auditory objects of increasing complexity. The dorsal stream engages in the time-dependent combination of elements, subserving both syntactic structuring and a linkage to action. Furthermore, frontal regions accomplish general aspects of cognitive control in the service of action planning and execution rather than linguistic processing. This architecture is supported by a range of existing empirical findings and helps to resolve a number of theoretical and empirical puzzles within the existing dorsal-ventral streams literature.

To Predict or Not to Predict: Influences of Task and Strategy on the Processing of Semantic Relations

We report a series of event-related potential experiments designed to dissociate the functionally distinct processes involved in the comprehension of highly restricted lexical-semantic relations (antonyms). We sought to differentiate between influences of semantic relatedness (which are independent of the experimental setting) and processes related to predictability (which differ as a function of the experimental environment). To this end, we conducted three ERP studies contrasting the processing of antonym relations (black-white) with that of related (black-yellow) and unrelated (black-nice) word pairs. Whereas the lexical-semantic manipulation was kept constant across experiments, the experimental environment and the task demands varied: Experiment 1 presented the word pairs in a sentence context of the form The opposite of X is Y and used a sensicality judgment. Experiment 2 used a word pair presentation mode and a lexical decision task. Experiment 3 also examined word pairs, but with an antonymy judgment task. All three experiments revealed a graded N400 response (unrelated > related > antonyms), thus supporting the assumption that semantic associations are processed automatically. In addition, the experiments revealed that, in highly constrained task environments, the N400 gradation occurs simultaneously with a P300 effect for the antonym condition, thus leading to the superficial impression of an extremely reduced N400 for antonym pairs. Comparisons across experiments and participant groups revealed that the P300 effect is not only a function of stimulus constraints (i.e., sentence context) and experimental task, but that it is also crucially influenced by individual processing strategies used to achieve successful task performance.

Neurobiological roots of language in primate audition: common computational properties.

Here, we present a new perspective on an old question: how does the neurobiology of human language relate to brain systems in nonhuman primates? We argue that higher-order language combinatorics, including sentence and discourse processing, can be situated in a unified, cross-species dorsal-ventral streams architecture for higher auditory processing, and that the functions of the dorsal and ventral streams in higher-order language processing can be grounded in their respective computational properties in primate audition. This view challenges an assumption, common in the cognitive sciences, that a nonhuman primate model forms an inherently inadequate basis for modeling higher-level language functions.

The role of the posterior superior temporal sulcus in the processing of unmarked transitivity

Is it living or not? The ability to differentiate between animate and inanimate entities is of considerable value in everyday life, since it allows for the dissociation of individuals that may willfully cause an action from objects that cannot. The present fMRI study aimed to shed light on the neural correlates of animacy at a relational-interpretive level, i.e. on the role of animacy in the establishment of relations between entities that are more or less likely to cause an event and differ in their potential to act volitionally. To this end, we investigated the processing of visually presented transitive German sentences (nominative-accusative structures) in which the factors animacy and argument order were manipulated. The relations between the arguments differed in that the animate subject either acted on an inanimate object (a very natural construction in terms of transitivity) or on an animate object (resulting in a sentence deviating from an unmarked transitive structure). Participants performed an acceptability judgment task. Violations of unmarked transitivity yielded a significant activation increase within the posterior left superior temporal sulcus (pSTS), thus suggesting a specific role of this cortical region in the relational use of animacy information. This result indicates that the influence of animacy as a relational feature differs from the impact of this parameter on the word level and is in line with other neuroimaging studies showing an engagement of the pSTS when a matching between syntax and semantics is required. A comparison between object- and subject-initial conditions further revealed a robust effect of argument order in the pars opercularis of the left inferior frontal gyrus (a subregion of Brocas area), thereby replicating previous findings demonstrating a sensitivity of this region to fine-grained language-specific linearization rules.

Two routes to actorhood: lexicalized potency to act and identification of the actor role.

The inference of causality is a crucial cognitive ability and language processing is no exception: recent research suggests that, across different languages, the human language comprehension system attempts to identify the primary causer of the state of affairs described (the “actor”) quickly and unambiguously (Bornkessel-Schlesewsky and Schlesewsky, 2009). This identification can take place verb-independently based on certain prominence cues (e.g., case, word order, animacy). Here, we present two experiments demonstrating that actor potential is also encoded at the level of individual nouns (a king is a better actor than a beggar). Experiment 1 collected ratings for 180 German nouns on 12 scales defined by adjective oppositions and deemed relevant for actorhood potential. By means of structural equation modeling, an actor potential (ACT) value was calculated for each noun. Experiment 2, an event-related potential study, embedded nouns from Experiment 1 in verb-final sentences, in which they were either actors or non-actors. N400 amplitude increased with decreasing ACT values and this modulation was larger for highly frequent nouns and for actor versus non-actor nouns. We argue that potency to act is lexically encoded for individual nouns and, since it modulates the N400 even for non-actor participants, it should be viewed as a property that modulates ease of lexical access (akin, for example, to lexical frequency). We conclude that two separate dimensions of actorhood computation are crucial to language comprehension: an experience-based, lexically encoded (bottom–up) representation of actorhood potential, and a prominence-based, computational mechanism for calculating goodness-of-fit to the actor role in a particular (top–down) sentence context.

Subjective Impressions Do Not Mirror Online Reading Effort: Concurrent EEG-Eyetracking Evidence from the Reading of Books and Digital Media

In the rapidly changing circumstances of our increasingly digital world, reading is also becoming an increasingly digital experience: electronic books (e-books) are now outselling print books in the United States and the United Kingdom. Nevertheless, many readers still view e-books as less readable than print books. The present study thus used combined EEG and eyetracking measures in order to test whether reading from digital media requires higher cognitive effort than reading conventional books. Young and elderly adults read short texts on three different reading devices: a paper page, an e-reader and a tablet computer and answered comprehension questions about them while their eye movements and EEG were recorded. The results of a debriefing questionnaire replicated previous findings in that participants overwhelmingly chose the paper page over the two electronic devices as their preferred reading medium. Online measures, by contrast, showed shorter mean fixation durations and lower EEG theta band voltage density – known to covary with memory encoding and retrieval – for the older adults when reading from a tablet computer in comparison to the other two devices. Young adults showed comparable fixation durations and theta activity for all three devices. Comprehension accuracy did not differ across the three media for either group. We argue that these results can be explained in terms of the better text discriminability (higher contrast) produced by the backlit display of the tablet computer. Contrast sensitivity decreases with age and degraded contrast conditions lead to longer reading times, thus supporting the conclusion that older readers may benefit particularly from the enhanced contrast of the tablet. Our findings thus indicate that peoples subjective evaluation of digital reading media must be dissociated from the cognitive and neural effort expended in online information processing while reading from such devices.

The processing of German word stress: evidence for the prosodic hierarchy

The present paper explores whether the metrical foot is necessary for the description of prosodic systems. To this end, we present empirical findings on the perception of German word stress using event-related brain potentials as the dependent measure. A manipulation of the main stress position within three-syllable words revealed differential brain responses, which (a) correlated with the reorganisation of syllables into feet in stress violations, and (b) differed in strength depending on syllable weight. The experiments therefore provide evidence that the processing of word stress not only involves lexical information about stress positions, but also (quantity-sensitive) information about metrical structures, in particular feet and syllables.

The P600-as-P3 hypothesis revisited: single-trial analyses reveal that the late EEG positivity following linguistically deviant material is reaction time aligned.

The P600, a late positive ERP component following linguistically deviant stimuli, is commonly seen as indexing structural, high-level processes, e.g. of linguistic (re)analysis. It has also been identified with the P3 (P600-as-P3 hypothesis), which is thought to reflect a systemic neuromodulator release facilitating behavioural shifts and is usually response time aligned. We investigated single-trial alignment of the P600 to response, a critical prediction of the P600-as-P3 hypothesis. Participants heard sentences containing morphosyntactic and semantic violations and responded via a button press. The elicited P600 was perfectly response aligned, while an N400 following semantic deviations was stimulus aligned. This is, to our knowledge, the first single-trial analysis of language processing data using within-sentence behavioural responses as temporal covariates. Results support the P600-as-P3 perspective and thus constitute a step towards a neurophysiological grounding of language-related ERPs.

Word order and Broca's region: evidence for a supra-syntactic perspective.

It has often been suggested that the role of Brocas region in sentence comprehension can be explained with reference to general cognitive mechanisms (e.g. working memory, cognitive control). However, the (language-related) basis for such proposals is often restricted to findings on English. Here, we argue that an extension of the database to other languages can shed new light on the types of mechanisms that an adequate account of Brocas region should be equipped to deal with. This becomes most readily apparent in the domain of word order variations, which we examined in German verb-final sentences using event-related fMRI. Our results showed that activation in the pars opercularis--a core subregion of Brocas area--was not only modulated by the relative ordering of subject and object, but also by a further factor known to affect word order in a number of languages, namely referentiality. Notably, the finding provides the first demonstration of a word order-related activation difference within subject-initial sentences in this region. Additional parametric analyses using individual behavioral data as predictors further attest to the independence of the pars opercularis activation from: (a) sentence acceptability, and (b) difficulty in performing the experimental (judgment) task. We argue that these and related findings attest to the need for a processing mechanism that can manipulate predicate-independent, interacting and hierarchically structured relational representations during real time comprehension. These properties pose a challenge to existing accounts of pars opercularis function.