Saskia van Dantzig

Perceptual Processing Affects Conceptual Processing

According to the Perceptual Symbols Theory of cognition (Barsalou, 1999), modality-specific simulations underlie the representation of concepts. A strong prediction of this view is that perceptual processing affects conceptual processing. In this study, participants performed a perceptual detection task and a conceptual property-verification task in alternation. Responses on the property-verification task were slower for those trials that were preceded by a perceptual trial in a different modality than for those that were preceded by a perceptual trial in the same modality. This finding of a modality-switch effect across perceptual processing and conceptual processing supports the hypothesis that perceptual and conceptual representations are partially based on the same systems.

Abstract concepts: sensory-motor grounding, metaphors, and beyond

textabstractAbstract In the last decade many researchers have obtained evidence for the idea that cognition shares processing mechanisms with perception and action. Most of the evidence supporting the grounded cognition framework focused on representations of concrete concepts, which leaves open the question how abstract concepts are grounded in sensory-motor processing. One promising idea is that people simulate concrete situations and introspective experiences to represent abstract concepts [Barsalou, L. W., & Wiemer-Hastings, K. (2005). Situating abstract concepts. In D. Pecher, & R. A. Zwaan (Eds.), Grounding cognition: The role of perception and action in memory, language, and thinking (pp. 129–163). Cambridge: Cambridge University Press.], although this has not yet been investigated a lot. A second idea, which more researchers have investigated, is that people use metaphorical mappings from concrete to abstract concepts [Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: Chicago University Press.]. According to this conceptual metaphor theory, image schemas structure and provide sensory-motor grounding for abstract concepts. Although there is evidence that people automatically activate image schemas when they process abstract concepts, we argue that situations are also needed to fully represent meaning.

Approach and avoidance as action effects

Numerous studies use arm movements (arm flexion and extension) to investigate the interaction between emotional stimuli and approach/avoidance behaviour. In many experiments, however, these arm movements are ambiguous. Arm flexion can be interpreted either as pulling (approach) or as withdrawing (avoidance). On the contrary, arm extension can be interpreted as reaching (approach) or as pushing (avoidance). This ambiguity can be resolved by regarding approach and avoidance as flexible action plans that are represented in terms of their effects. Approach actions reduce the distance between a stimulus and the self, whereas avoidance actions increase that distance. In this view, action effects are an integral part of the representation of an action. As a result, a neutral action can become an approach or avoidance reaction if it repeatedly results in decreasing or increasing the distance to a valenced stimulus. This hypothesis was tested in the current study. Participants responded to positive and negative words using key-presses. These “neutral” responses (not involving arm flexion or extension) were consistently followed by a stimulus movement toward or away from the participant. Responses to emotional words were faster when the responses effect was congruent with stimulus valence, suggesting that approach/avoidance actions are indeed defined in terms of their outcomes.

Toward a persuasive mobile application to reduce sedentary behavior

Prolonged sitting is a potential health risk, not only for people with an inactive lifestyle but also for those who meet the daily physical activity recommendations. Mobile applications that trigger people to take regular breaks from sitting seem promising. In this paper, we present the results of our quest to create effective persuasive mobile applications aimed at reducing sedentary behavior. First, we developed SitCoach, a mobile application to nudge office workers from their seats. SitCoach monitors physical activity and sedentary behavior and provides timely persuasive messages suggesting active breaks. A user test showed that users had little awareness of the risks of prolonged sitting and considered their ability to take active breaks to be highly dependent on external factors. The results from this study formed the basis for a second experiment, which was more extensive in duration and number of participants. In this 6-week experiment, office workers received timely persuasive messages on their smart phones, advising them to take an active break whenever they were sitting behind their computer for too long. Compared to a Control group who did not receive these messages, a significant decrease in computer activity was achieved. The studies show the potential and limitations of using a smart phone as a platform for reducing sedentary behavior. We conclude with recommendations to create effective mobile applications that motivate people to take regular breaks from sitting.

Language comprehenders retain implied shape and orientation of objects

According to theories of embodied cognition, language comprehenders simulate sensorimotor experiences to represent the meaning of what they read. Previous studies have shown that picture recognition is better if the object in the picture matches the orientation or shape implied by a preceding sentence. In order to test whether strategic imagery may explain previous findings, language comprehenders first read a list of sentences in which objects were mentioned. Only once the complete list had been read was recognition memory tested with pictures. Recognition performance was better if the orientation or shape of the object matched that implied by the sentence, both immediately after reading the complete list of sentences and after a 45-min delay. These results suggest that previously found match effects were not due to strategic imagery and show that details of sensorimotor simulations are retained over longer periods.

Congruency between Word Position and Meaning is Caused by Task-Induced Spatial Attention

We report an experiment that compared two explanations for the effect of congruency between a words on screen spatial position and its meaning. On one account, congruency is explained by the match between position and a mental simulation of meaning. Alternatively, congruency is explained by the polarity alignment principle. To distinguish between these accounts we presented the same object names (e.g., shark, helicopter) in a sky decision task or an ocean decision task, such that response polarity and typical location were disentangled. Sky decision responses were faster to words at the top of the screen compared to words at the bottom of the screen, but the reverse was found for ocean decision responses. These results are problematic for the polarity principle, and support the claim that spatial attention is directed by mental simulation of the task-relevant conceptual dimension.

A sharp image or a sharp knife: norms for the modality-exclusivity of 774 concept-property items

According to recent embodied cognition theories, mental concepts are represented by modality-specific sensory-motor systems. Much of the evidence for modality-specificity in conceptual processing comes from the property-verification task. When applying this and other tasks, it is important to select items based on their modality-exclusivity. We collected modality ratings for a set of 387 properties, each of which was paired with two different concepts, yielding a total of 774 concept-property items. For each item, participants rated the degree to which the property could be experienced through five perceptual modalities (vision, audition, touch, smell, and taste). Based on these ratings, we computed a measure of modality exclusivity, the degree to which a property is perceived exclusively through one sensory modality. In this paper, we briefly sketch the theoretical background of conceptual knowledge, discuss the use of the property-verification task in cognitive research, provide our norms and statistics, and validate the norms in a memory experiment. We conclude that our norms are important for researchers studying modality-specific effects in conceptual processing.

A computational model of perception and action for cognitive robotics.

Robots are increasingly expected to perform tasks in complex environments. To this end, engineers provide them with processing architectures that are based on models of human information processing. In contrast to traditional models, where information processing is typically set up in stages (i.e., from perception to cognition to action), it is increasingly acknowledged by psychologists and robot engineers that perception and action are parts of an interactive and integrated process. In this paper, we present HiTEC, a novel computational (cognitive) model that allows for direct interaction between perception and action as well as for cognitive control, demonstrated by task-related attentional influences. Simulation results show that key behavioral studies can be readily replicated. Three processing aspects of HiTEC are stressed for their importance for cognitive robotics: (1) ideomotor learning of action control, (2) the influence of task context and attention on perception, action planning, and learning, and (3) the interaction between perception and action planning. Implications for the design of cognitive robotics are discussed.

Concepts are not represented by conscious imagery

According to theories of grounded cognition, conceptual representation and perception share processing mechanisms. We investigated whether this overlap is due to conscious perceptual imagery. Participants filled out questionnaires to assess the vividness of their imagery (Questionnaire on Mental Imagery) and the extent to which their imagery was object oriented and spatially oriented (Object-Spatial Imagery Questionnaire), and they performed a mental rotation task. One week later, they performed a verbal property verification task. In this task, involvement of modality-specific systems is evidenced by the modality-switch effect, the finding that performance on a target trial (e.g., apple—green) is better after a same-modality trial (e.g., diamond—sparkle) than after a different-modality trial (e.g., airplane—noisy). Results showed a modality-switch effect, but there was no systematic relation between imagery scores and modality switch. We conclude that conscious mental imagery is not fundamental to conceptual representation.

How Task Goals Mediate the Interplay between Perception and Action

Theories of embodied cognition suppose that perception, action, and cognition are tightly intertwined and share common representations and processes. Indeed, numerous empirical studies demonstrate interaction between stimulus perception, response planning, and response execution. In this paper, we present an experiment and a connectionist model that show how the Simon effect, a canonical example of perception–action congruency, can be moderated by the (cognitive representation of the) task instruction. To date, no representational account of this influence exists. In the experiment, a two-dimensional Simon task was used, with critical stimuli being colored arrows pointing in one of four directions (backward, forward, left, or right). Participants stood on a Wii balance board, oriented diagonally toward the screen displaying the stimuli. They were either instructed to imagine standing on a snowboard or on a pair of skis and to respond to the stimulus color by leaning toward either the left or right foot. We expected that participants in the snowboard condition would encode these movements as forward or backward, resulting in a Simon effect on this dimension. This was confirmed by the results. The left–right congruency effect was larger in the ski condition, whereas the forward–backward congruency effect appeared only in the snowboard condition. The results can be readily accounted for by HiTEC, a connectionist model that aims at capturing the interaction between perception and action at the level of representations, and the way this interaction is mediated by cognitive control. Together, the empirical work and the connectionist model contribute to a better understanding of the complex interaction between perception, cognition, and action.