Michele Burigo

Visual attention during spatial language comprehension.

Spatial terms such as “above”, “in front of”, and “on the left of” are all essential for describing the location of one object relative to another object in everyday communication. Apprehending such spatial relations involves relating linguistic to object representations by means of attention. This requires at least one attentional shift, and models such as the Attentional Vector Sum (AVS) predict the direction of that attention shift, from the sausage to the box for spatial utterances such as “The box is above the sausage”. To the extent that this prediction generalizes to overt gaze shifts, a listener’s visual attention should shift from the sausage to the box. However, listeners tend to rapidly look at referents in their order of mention and even anticipate them based on linguistic cues, a behavior that predicts a converse attentional shift from the box to the sausage. Four eye-tracking experiments assessed the role of overt attention in spatial language comprehension by examining to which extent visual attention is guided by words in the utterance and to which extent it also shifts “against the grain” of the unfolding sentence. The outcome suggests that comprehenders’ visual attention is predominantly guided by their interpretation of the spatial description. Visual shifts against the grain occurred only when comprehenders had some extra time, and their absence did not affect comprehension accuracy. However, the timing of this reverse gaze shift on a trial correlated with that trial’s verification time. Thus, while the timing of these gaze shifts is subtly related to the verification time, their presence is not necessary for successful verification of spatial relations.

Context Affects Scale Selection for Proximity Terms

Abstract Spatial proximity terms, such as near and far, communicate information regarding the distance in which a “located” object can be found with respect to a “reference” object. The present paper investigates whether people take into account the location of an object extraneous to the located object and reference object pair, when setting the scale for proximity language judgements. Across three experiments participants rated the appropriateness of near and far to describe spatial scenes that included a third (distractor) object positioned the same distance as the located object from the reference object, but at varying distances from the located object. The results show that the presence of other spatial relations affects scale setting, resulting in differences in appropriateness ratings for those spatial terms.

Spatial language and converseness

Typical spatial language sentences consist of describing the location of an object (the located object) in relation to another object (the reference object) as in “The book is above the vase”. While it has been suggested that the properties of the located object (the book) are not translated into language because they are irrelevant when exchanging location information, it has been shown that the orientation of the located object affects the production and comprehension of spatial descriptions. In line with the claim that spatial language apprehension involves inferences about relations that hold between objects it has been suggested that during spatial language apprehension people use the orientation of the located object to evaluate whether the logical property of converseness (e.g., if “the book is above the vase” is true, then also “the vase is below the book” must be true) holds across the objects’ spatial relation. In three experiments using sentence acceptability rating tasks we tested this hypothesis and demonstrated that when converseness is violated peoples acceptability ratings of a scenes description are reduced indicating that people do take into account geometric properties of the located object and use it to infer logical spatial relations.

Inference-driven attention in symbolic and perceptual tasks: Biases toward expected and unexpected inputs:

The aims of this paper are (a) to gather support for the hypothesis that some basic mechanisms of attentional deployment (i.e., its high efficiency in dealing with expected and unexpected inputs) meet the requirements of the inferential system and have possibly evolved to support its functioning, and (b) to show that these orienting mechanisms function in very similar ways in two perceptual tasks and in a symbolic task. The general hypothesis and its predictions are sketched in the Introduction, after a discussion of current findings concerning visual attention and the generalities of the inferential system. In the empirical section, three experiments are presented where participants tracked visual trajectories (Experiments 1 and 3) or arithmetic series (Experiments 2 and 3), responding to the onset of a target event (e.g., to a specific number) and to the repetition of an event (e.g., to a number appearing twice consecutively). Target events could be anticipated when they were embedded in regular series/trajectories; they could be anticipated, with the anticipation later disconfirmed, when a regular series/trajectory was abruptly interrupted before the target event occurred; and they could not be anticipated when the series/trajectory was random. Repeated events could not be anticipated. Results show a very similar pattern of allocation in tracking visual trajectories and arithmetic series: Attention is focused on anticipated events; it is defocused and redistributed when an anticipation is not confirmed by ensuing events; however, performance decreases when dealing with random series/trajectory—that is, in the absence of anticipations. In our view, this is due to the fact that confirmed and disconfirmed anticipations are crucial events for “knowledge revision”—that is, the fine tuning of the inferential system to the environment; attentional mechanisms have developed so as to enhance detection of these events, possibly at all levels of inferential processing.

Working memory in healthy aging and in Parkinson’s disease: evidence of interference effects

ABSTRACT Focusing on relevant information while suppressing the irrelevant one are critical abilities for different cognitive processes. However, their functioning has been scarcely investigated in the working memory (WM) domain, in both healthy and pathological conditions. The present research aimed to study these abilities in aging and Parkinson’s disease (PD), testing three groups of healthy participants (young, older and elderly) and one of PD patients, employing a new experimental paradigm. Results showed that the transient storing of irrelevant information in WM causes substantial interference effects, which were remarkable in elderly individuals on both response latency and accuracy. Interestingly, PD patients responded faster and were equally accurate compared to a matched control group. Taken together, findings confirm the existence of similar mechanisms for orienting attention inwards to WM contents or outwards to perceptual stimuli, and suggest the suitability of our task to assess WM functioning in both healthy aging and PD.

Shifts of Attention During Spatial Language Comprehension

Regier and Carlson (2001) have investigated the processing of spatial prepositions and developed a cognitive model that formalizes how spatial prepositions are evaluated against depicted spatial relations between objects. In their Attentional Vector Sum (AVS) model, a population of vectors is weighted with visual attention, rooted at the reference object and pointing to the located object. The deviation of the vector sum from a reference direction is then used to evaluate the goodness-of-fit of the spatial preposition. Crucially, the AVS model assumes a shift of attention from the reference object to the located object. The direction of this shift has been challenged by recent psycholinguistic and neuroscientific findings. We propose a modified version of the AVS model (the rAVS model) that integrates these findings. In the rAVS model, attention shifts from the located object to the reference object in contrast to the attentional shift from the reference object to the located object implemented in the AVS model. Our model simulations show that the rAVS model accounts for both the data that inspired the AVS model and the most recent findings.

Processing Sentences with Literal versus Figurative Use of Verbs: An ERP Study with Children with Language Impairments, Nonverbal Impairments, and Typical Development

Forty native Italian children (age 6–15) performed a sentence plausibility judgment task. ERP recordings were available for 12 children with specific language impairment (SLI), 11 children with nonverbal learning disabilities (NVLD), and 13 control children. Participants listened to verb-object combinations and judged them as acceptable or unacceptable. Stimuli belonged to four conditions, where concreteness and congruency were manipulated. All groups made more errors responding to abstract and to congruent sentences. Moreover, SLI participants performed worse than NVLD participants with abstract sentences. ERPs were analyzed in the time window 300–500 ms. SLI children show atypical, reversed effects of concreteness and congruence as compared to control and NVLD children, respectively. The results suggest that linguistic impairments disrupt abstract language processing more than visual-motor impairments. Moreover, ROI and SPM analyses of ERPs point to a predominant involvement of the left rather than the right hemisphere in the comprehension of figurative expressions.

Reference frame conflict in assigning direction to space

Spatial prepositions are linguistic tools to exchange information about spatial location of objects. For instance “The book is over the table” indicates that the located object (LO) is somewhere “over” the reference object (RO). Assigning direction to space (selecting a reference frame) is a necessary precursor to understanding where the LO is located. Three experiments are reported which investigated the effect of the orientation of both the LO and the RO on the acceptability of the prepositions above/below/over/under. We found that when the LO was not vertically aligned, the appropriateness for a given spatial preposition changes. In general scenes with the LO pointing at the RO were judged less acceptable than scenes with the LO vertically oriented. These results suggest that people generate reference frames for both LO and RO prior to assigning direction to space. Modifications to Multiple Frame Activation theory [1] are discussed.

Modeling the Directionality of Attention During Spatial Language Comprehension

It is known that the comprehension of spatial prepositions involves the deployment of visual attention. For example, consider the sentence “The salt is to the left of the stove”. Researchers [29, 30] have theorized that people must shift their attention from the stove (the reference object, RO) to the salt (the located object, LO) in order to comprehend the sentence. Such a shift was also implicitly assumed in the Attentional Vector Sum (AVS) model by [35], a cognitive model that computes an acceptability rating for a spatial preposition given a display that contains an RO and an LO. However, recent empirical findings showed that a shift from the RO to the LO is not necessary to understand a spatial preposition ([3], see also [15, 38]). In contrast, these findings suggest that people perform a shift in the reverse direction (i.e., from the LO to the RO). Thus, we propose the reversed AVS (rAVS) model, a modified version of the AVS model in which attention shifts from the LO to the RO. We assessed the AVS and the rAVS model on the data from [35] using three model simulation methods. Our simulations show that the rAVS model performs as well as the AVS model on these data while it also integrates the recent empirical findings. Moreover, the rAVS model achieves its good performance while being less flexible than the AVS model. (This article is an updated and extended version of the paper [23] presented at the 8th International Conference on Agents and Artificial Intelligence in Rome, Italy. The authors would like to thank Holger Schultheis for helpful discussions about the additional model simulation.)

Learning and Using Abstract Words: Evidence from Clinical Populations

It has been shown that abstract concepts are more difficult to process and are acquired later than concrete concepts. We analysed the percentage of concrete words in the narrative lexicon of individuals with Williams Syndrome (WS) as compared to individuals with Down Syndrome (DS) and typically developing (TD) peers. The cognitive profile of WS is characterized by visual-spatial difficulties, while DS presents with predominant impairments in linguistic abilities. We predicted that if linguistic abilities are crucial to the development and use of an abstract vocabulary, DS participants should display a higher concreteness index than both Williams Syndrome and typically developing individuals. Results confirm this prediction, thus supporting the hypothesis of a crucial role of linguistic processes in abstract language acquisition. Correlation analyses suggest that a maturational link exists between the level of abstractness in narrative production and syntactic comprehension.