Kenny R. Coventry

Spatial prepositions, object-specific function, and task requirements

Two separate issues were looked at in this experimental study of the semantics of spatial prepositions. In the context of work to specify general factors of a functional geometry mediating the use of spatial prepositions (Garrod & Sanford 1989; Coventry 1992, 1993), object-specific effects were investigated. Subjects described video scenes of various objects and their responses of in, on, over, and beside were monitored. The independent variables involved the manipulation of functionality specific to various types of objects. It was concluded that knowledge about how particular objects interact with each other contributes to the representation of functional relations which determine preposition usage. Therefore a specification of functional geometries cannot proceed without a prior formulation of our knowledge about the physical and social worlds. Additionally two different experimental measures of prepositional covariance with the scenes were used: Lickert-scale judgements and sentence completions. Responses from two separate groups were compared. The findings indicated some agreement between the two measures, but also some differences in patterns of response. It is suggested that the measures are tapping different processes, and that a variety of methods need to be used to abstract to lexical representation.

Predictive Eye Movements Are Driven by Goals, Not by the Mirror Neuron System

The importance of a mirror neuron system (MNS) as a mechanism for understanding the actions of others has been established (e.g., Rizzolatti, Fadiga, Gallese, & Fogassi, 1996). Neurons in the primate premotor cortex fire both when a monkey performs an action (e.g., grasping) and when the monkey observes someone else performing the same action. It has also been proposed that the MNS is the starting point for understanding the intentions and goal-directed behavior of others (Fogassi et al., 2005; Gallese & Goldman, 1998). Consistent with this view, Falck-Ytter, Gredeback, and von Hofsten (2006) argued that the MNS is implicated in proactive goal-directed (predictive) eye movements. In a series of eye-tracking studies, participants observed a toy object moving along a trajectory toward a container. Adults and 1-year-old infants looked ahead of the toy and toward the goal container only when a hand was observed moving the toy (the human-agent condition). Proactive goaldirected eye movements were not found in two further conditions: a self-propelled condition, in which a toy with rudimentary facial features moved along the trajectory by itself, or a mechanical motion condition, in which a ball with no distinctive features moved along the trajectory. Falck-Ytter et al. interpreted these data as evidence that the MNS is necessary for proactive goal-directed eye movements. Moreover, the absence of proactive goal-directed eye movements in 6-month-old infants (who are too young to perform the actions themselves) is taken as further support for the MNS account. These claims are premature. The conditions run by FalckYtter et al. (2006) do not discriminate between predicted human motion tied to intention and predicted agent goals tied to intention. Consistent with the mirror neuron hypothesis, movement of the hand may necessarily involve the simulation of motion via the MNS, and proactive goal-directed eye movements may therefore only occur when a hand is shown to move the object. Alternatively, a hand moving an object involves the intention of an agent to place the object in the goal container, and the expectation that the agent intends for the object to end up in a goal location may cause the proactive eye movements (consistent with teleological stance theory; Gergely & Csibra, 2003). To arbitrate between these accounts, it is necessary to run a human-agent condition without human movement. Therefore, we set out to test the claims of Falck-Ytter, et al. (2006), but with the addition of a new critical condition missing in the original study. We ran three movement conditions: the human-agent condition, in which a human agent was shown moving a toy frog toward a goal container (i.e., [1human agent, 1human motion]); the self-propelled condition, in which no human agent was shown moving the frog (i.e., [ human agent, human motion]); and the new condition, in which a human agent was shown with hand behind the starting point of the frog, flicking it so as to propel it along a trajectory (as in the game ‘‘Tiddlywinks’’; i.e., [1human agent, human motion]; see Fig. 1a). In the latter condition, the human-agent intention is matched to that of the human-agent condition, but human motion is not shown along the trajectory. This allows a clean test of the MNS versus goal-intention explanations for the proactive eye-movement data. We also ran each condition in two ways. In the original humanagent condition run by Falck-Ytter et al. (2006), when the toy object reached the bucket, a sound was played and a smiley face on the bucket was animated. This could serve to heighten the desirability of the goal state, encouraging proactive eye movements in a manner consistent with teleological stance theory. Therefore, we ran each of the three conditions with and without end effects. For the conditions with end effects, when the toy The order of authorship is arbitrary; all authors contributed equally to the work reported. Address correspondence to Kenny R. Coventry, Cognition and Communication Research Centre, Northumbria University, Northumberland Building, Northumberland Rd., Newcastle Upon Tyne NE1 8ST, United Kingdom, e-mail: kenny.coventry@ northumbria.ac.uk. PSYCHOLOGICAL SCIENCE

Spatial language, visual attention, and perceptual simulation

Spatial language descriptions, such as The bottle is over the glass, direct the attention of the hearer to particular aspects of the visual world. This paper asks how they do so, and what brain mechanisms underlie this process. In two experiments employing behavioural and eye tracking methodologies we examined the effects of spatial language on peoples judgements and parsing of a visual scene. The results underscore previous claims regarding the importance of object function in spatial language, but also show how spatial language differentially directs attention during examination of a visual scene. We discuss implications for existing models of spatial language, with associated brain mechanisms.

Mood state and gambling: using mobile telephones to track emotions.

Mobile telephones were used to collect data on the relationship between gambling and mood state from gamblers in the field. Seventeen gamblers called an interactive voice response system running on a computer before, during and after a gambling episode. Measures taken in this way included self-reports of anxiety/arousal, the amount of money gambled, whether the result was a win or loss, the amount won or lost, and the type of gambling engaged in. Other measures were taken during an initial briefing session using conventional questionnaires that included self-reports of anxiety/arousal taken in a non-gambling situation, dissociation during gambling, and a measure of degree of impairment of control. The results showed that subjective anxiety/arousal levels were significantly higher during and after gambling than during the urge to gamble or at baselines. Losing was associated with increased subjective anxiety/arousal after play, and winning was associated with a decrease in subjective anxiety/arousal. This suggests that gambling may be a cause of increased subjective anxiety/arousal, rather than functioning to relieve it. A cluster of variables associated with impaired control and subjective anxiety/arousal levels was also identified. The method of collecting data using mobile telephones appears to be a valuable development.

Spatial prepositions and vague quantifiers: implementing the functional geometric framework

There is much empirical evidence showing that factors other than the relative positions of objects in Euclidean space are important in the comprehension of a wide range of spatial prepositions in English and other languages. We first the overview the functional geometric framework [11] which puts “what” and “where” information together to underpin the situation specific meaning of spatial terms. We then outline an implementation of this framework. The computational model for the processing of visual scenes and the identification of the appropriate spatial preposition consists of three main modules: (1) Vision Processing, (2) Elman Network, (3) Dual-Route Network. Mirroring data from experiments with human participants, we show that the model is both able to predict what will happen to objects in a scene, and use these judgements to influence the appropriateness of over/under/above/below to describe where objects are located in the scene. Extensions of the model to other prepositions and quantifiers are discussed.

On the ups and downs of emotion:testing between conceptual-metaphor and polarity accounts of emotional valence–spatial location interactions

In the past decade, many studies have focused on the relationship between emotional valence and vertical spatial positions from a processing perspective. Lakoff and Johnson’s (1980) work on conceptual metaphor has traditionally motivated these investigations, but recent work (Lakens in J Exp Psychol: Learn, Mem Cogn, 38: 726–736, 2012) has suggested that polarity-based perspectives offer an alternative account of response time patterns. We contrasted the predictions of these two theories using a new facial emotion recognition task, in which participants made speeded responses to happy or sad faces on a display, with the spatial location of those faces being manipulated. In three experiments (two-alternative forced choice tasks and a go/no-go task), we found a pattern of responses consistent with a polarity-based account, but inconsistent with key predictions of the conceptual-metaphor account. Overall, congruency effects were observed for positively valenced items, but not for negatively valenced items. These findings demonstrate that polarity effects extend to nonlinguistic stimuli and beyond two-alternative forced choice tasks. We discuss the results in terms of common-coding approaches to task–response mappings.

Function, geometry and spatial prepositions: Threeexperiments

In this paper the results of three experiments are reported whichaddress the issue of the relative extent to which functionalrelations versus geometric relations affect spatial language. Theexperiments examine the role of a discourse context on the useand rating of the preposition in to describe a visual scenewhere the constraint of spatial containment between figure(object located) and ground (reference object) does not hold. Allthree experiments demonstrate that in is used more andrated to be significantly more appropriate in a functionalcontext than in a no context condition. The implications of thesestudies for spatial language and spatial representation arediscussed.

Spatial demonstratives and perceptual space: describing and remembering object location.

Spatial demonstratives - terms including this and that - are among the most common words across all languages. Yet, there are considerable differences between languages in how demonstratives carve up space and the object characteristics they can refer to, challenging the idea that the mapping between spatial demonstratives and the vision and action systems is universal. In seven experiments we show direct parallels between spatial demonstrative usage in English and (non-linguistic) memory for object location, indicating close connections between the language of space and non-linguistic spatial representation. Spatial demonstrative choice in English and immediate memory for object location are affected by a range of parameters - distance, ownership, visibility and familiarity - that are lexicalized in the demonstrative systems of some other languages. The results support a common set of constraints on language used to talk about space and on (non-linguistic) spatial representation itself. Differences in demonstrative systems across languages may emerge from basic distinctions in the representation and memory for object location. In turn, these distinctions offer a building block from which non-spatial uses of demonstratives can develop.

Object-specific function, geometry, and the comprehension of in and on

Three experiments are reported which examined the relative effects of geometry and object-specific function on the comprehension of the spatial prepositions in and on. The first experiment manipulated the height of a located object on top of a pile of other objects in containers which were primarily containers of solids (e.g., a suitcase) or liquids (e.g., an aquarium). The association between located object and reference object was also varied (by using different types of objects as located objects). In was found to be more appropriate to describe the same object in containers of solids compared to containers of liquids, although no effects of located object association were found. Experiments 2 and 3 manipulated similar variables with supporting surfaces rather than containers, and examined the effects of functional control on the comprehension of on. The studies provide evidence for the importance of functional relations on the comprehension of on. In addition effects of located object association were found, but only when there was no clear evidence for the absence or presence of functional control. The results are discussed in relation to the differential effects of object knowledge on the comprehension of spatial prepositions.

Multiple Routes to Mental Animation Language and Functional Relations Drive Motion Processing for Static Images

When looking at static visual images, people often exhibit mental animation, anticipating visual events that have not yet happened. But what determines when mental animation occurs? Measuring mental animation using localized brain function (visual motion processing in the middle temporal and middle superior temporal areas, MT+), we demonstrated that animating static pictures of objects is dependent both on the functionally relevant spatial arrangement that objects have with one another (e.g., a bottle above a glass vs. a glass above a bottle) and on the linguistic judgment to be made about those objects (e.g., “Is the bottle above the glass?” vs. “Is the bottle bigger than the glass?”). Furthermore, we showed that mental animation is driven by functional relations and language separately in the right hemisphere of the brain but conjointly in the left hemisphere. Mental animation is not a unitary construct; the predictions humans make about the visual world are driven flexibly, with hemispheric asymmetry in the routes to MT+ activation.