Jorrig Vogels

Who is where referred to how, and why? : The influence of visual saliency on referent accessibility in spoken language production

Salient entities are assumed to be more accessible in memory, which makes them more likely to be referred to first and to be referred to with an attenuated expression, such as a pronoun. It is less clear, however, how different types of salience interact in influencing referent accessibility. In this article, we address the question whether non-linguistic factors can affect accessibility in the presence of a linguistic context. We present two story completion experiments in which we investigated the effect of visual salience (foregrounding) in interaction with linguistic salience (subjecthood) of two story characters both on the choice of referent and on the choice of referring expression. In Experiment 1, linguistic salience was moderated by inducing a topic shift in the discourse context. In Experiment 2, contexts in which linguistic salience was unclear were compared to contexts in which one of the characters was highly linguistically salient. The results show that visual salience influences referent choice independently of linguistic salience, but that it does not have an effect on the choice of referring expression. This suggests that visual salience has an influence on the global interpretation of the scene, but does not directly affect the accessibility status of individual entities. This is compatible with a view of language production in which utterance planning is influenced by conceptual and discourse factors rather than by low-level perceptual factors.

When a Stone Tries to Climb up a Slope: The Interplay between Lexical and Perceptual Animacy in Referential Choices

Several studies suggest that referential choices are influenced by animacy. On the one hand, animate referents are more likely to be mentioned as subjects than inanimate referents. On the other hand, animate referents are more frequently pronominalized than inanimate referents. These effects have been analyzed as effects of conceptual accessibility. In this paper, we raise the question whether these effects are driven only by lexical concepts, such that referents described by animate lexical items (e.g., “toddler”) are more accessible than referents described by inanimate lexical items (e.g., “shoe”), or can also be influenced by context-derived conceptualizations, such that referents that are perceived as animate in a particular context are more accessible than referents that are not. In two animation-retelling experiments, conducted in Dutch, we investigated the influence of lexical and perceptual animacy on the choice of referent and the choice of referring expression. If the effects of animacy are context-dependent, entities that are perceived as animate should yield more subject references and more pronouns than entities that are perceived as inanimate, irrespective of their lexical animacy. If the effects are tied to lexical concepts, entities described with animate lexical items should be mentioned as the subject and pronominalized more frequently than entities described with inanimate lexical items, irrespective of their perceptual animacy. The results show that while only lexical animacy appears to affect the choice of subject referent, perceptual animacy may overrule lexical animacy in the choice of referring expression. These findings suggest that referential choices can be influenced by conceptualizations based on the perceptual context.

Salience and Attention in Surprisal-Based Accounts of Language Processing

The notion of salience has been singled out as the explanatory factor for a diverse range of linguistic phenomena. In particular, perceptual salience (e.g., visual salience of objects in the world, acoustic prominence of linguistic sounds) and semantic-pragmatic salience (e.g., prominence of recently mentioned or topical referents) have been shown to influence language comprehension and production. A different line of research has sought to account for behavioral correlates of cognitive load during comprehension as well as for certain patterns in language usage using information-theoretic notions, such as surprisal. Surprisal and salience both affect language processing at different levels, but the relationship between the two has not been adequately elucidated, and the question of whether salience can be reduced to surprisal / predictability is still open. Our review identifies two main challenges in addressing this question: terminological inconsistency and lack of integration between high and low levels of representations in salience-based accounts and surprisal-based accounts. We capitalize upon work in visual cognition in order to orient ourselves in surveying the different facets of the notion of salience in linguistics and their relation with models of surprisal. We find that work on salience highlights aspects of linguistic communication that models of surprisal tend to overlook, namely the role of attention and relevance to current goals, and we argue that the Predictive Coding framework provides a unified view which can account for the role played by attention and predictability at different levels of processing and which can clarify the interplay between low and high levels of processes and between predictability-driven expectation and attention-driven focus.

How Cognitive Load Influences Speakers' Choice of Referring Expressions

We report on two experiments investigating the effect of an increased cognitive load for speakers on the choice of referring expressions. Speakers produced story continuations to addressees, in which they referred to characters that were either salient or non-salient in the discourse. In Experiment 1, referents that were salient for the speaker were non-salient for the addressee, and vice versa. In Experiment 2, all discourse information was shared between speaker and addressee. Cognitive load was manipulated by the presence or absence of a secondary task for the speaker. The results show that speakers under load are more likely to produce pronouns, at least when referring to less salient referents. We take this finding as evidence that speakers under load have more difficulties taking discourse salience into account, resulting in the use of expressions that are more economical for themselves.

Where to place inaccessible subjects in Dutch: The role of definiteness and animacy

Abstract Cross-linguistically, both subjects and topical information tend to be placed at the beginning of a sentence. Subjects are generally highly topical, causing both tendencies to converge on the same word order. However, subjects that lack prototypical topic properties may give rise to an incongruence between the preference to start a sentence with the subject and the preference to start a sentence with the most accessible information. We present a corpus study in which we investigate in what syntactic position (preverbal or postverbal) such low-accessible subjects are typically found in Dutch natural language. We examine the effects of both discourse accessibility (definiteness) and inherent accessibility (animacy). Our results show that definiteness and animacy interact in determining subject position in Dutch. Non-referential (bare) subjects are less likely to occur in preverbal position than definite subjects, and this tendency is reinforced when the subject is inanimate. This suggests that these two properties that make the subject less accessible together can ‘gang up’ against the subject first preference. The results support a probabilistic multifactorial account of syntactic variation.