M. van Elk

Action semantics: A unifying conceptual framework for the selective use of multimodal and modality-specific object knowledge

Our capacity to use tools and objects is often considered one of the hallmarks of the human species. Many objects greatly extend our bodily capabilities to act in the physical world, such as when using a hammer or a saw. In addition, humans have the remarkable capability to use objects in a flexible fashion and to combine multiple objects in complex actions. We prepare coffee, cook dinner and drive our car. In this review we propose that humans have developed declarative and procedural knowledge, i.e. action semantics that enables us to use objects in a meaningful way. A state-of-the-art review of research on object use is provided, involving behavioral, developmental, neuropsychological and neuroimaging studies. We show that research in each of these domains is characterized by similar discussions regarding (1) the role of object affordances, (2) the relation between goals and means in object use and (3) the functional and neural organization of action semantics. We propose a novel conceptual framework of action semantics to address these issues and to integrate the previous findings. We argue that action semantics entails both multimodal object representations and modality-specific sub-systems, involving manipulation knowledge, functional knowledge and representations of the sensory and proprioceptive consequences of object use. Furthermore, we argue that action semantics are hierarchically organized and selectively activated and used depending on the action intention of the actor and the current task context. Our framework presents an integrative account of multiple findings and perspectives on object use that may guide future studies in this interdisciplinary domain.

Semantics in action: An electrophysiological study on the use of semantic knowledge for action

Recent studies have supported close interactions between language and action-related processes, suggesting comparable neural mechanisms. However, relatively little is known about the semantics involved in action planning. The present study investigated the activation of semantic knowledge in meaningful actions by recording event-related potentials (ERPs). Subjects prepared meaningful or meaningless actions with objects and made a semantic categorization response before executing the action. Words presented could be either congruent or incongruent with respect to the goal of the action. Preparation of meaningful actions elicited a larger anterior N400 for words incongruent to the present action goal as compared to congruent words, while no N400 effect was found when subjects prepared meaningless actions. These findings indicate that the preparation of meaningful actions with objects is accompanied by the activation of semantic information representing the usual action goals associated with those objects.

Conceptual knowledge for understanding other’s actions is organized primarily around action goals

Semantic knowledge about objects entails both knowing how to grasp an object (grip-related knowledge) and what to do with an object (goal-related knowledge). Considerable evidence suggests a hierarchical organization in which specific hand-grips in action execution are most often selected to accomplish a remote action goal. The present study aimed to investigate whether a comparable hierarchical organization of semantic knowledge applies to the recognition of other’s object-directed actions as well. Correctness of either the Grip (hand grip applied to the object) or the Goal (end-location at which an object was directed) were manipulated independently in two experiments. In Experiment 1, subjects were required to attend selectively to either the correctness of the grip or the goal of the observed action. Subjects were faster when attending to the goal of the action and a strong interference of goal-violations was observed when subjects attended to the grip of the action. Importantly, observation of irrelevant goal- or grip-related violations interfered with making decisions about the correctness of the relevant dimension only when the relevant dimension was correct. In contrast, in Experiment 2, when subjects attended to an action-irrelevant stimulus dimension (i.e. orientation of the object), no interference of goal- or grip-related violations was found, ruling out the possibility that interference-effects result from perceptual differences between stimuli. These findings suggest that understanding the correctness of an action selectively recruits specialized, but interacting networks, processing the correctness of goal- and grip-specific information during action observation.

Meta-analyses are no substitute for registered replications: a skeptical perspective on religious priming

According to a recent meta-analysis, religious priming has a positive effect on prosocial behavior (Shariff et al., 2015). We first argue that this meta-analysis suffers from a number of methodological shortcomings that limit the conclusions that can be drawn about the potential benefits of religious priming. Next we present a re-analysis of the religious priming data using two different meta-analytic techniques. A Precision-Effect Testing–Precision-Effect-Estimate with Standard Error (PET-PEESE) meta-analysis suggests that the effect of religious priming is driven solely by publication bias. In contrast, an analysis using Bayesian bias correction suggests the presence of a religious priming effect, even after controlling for publication bias. These contradictory statistical results demonstrate that meta-analytic techniques alone may not be sufficiently robust to firmly establish the presence or absence of an effect. We argue that a conclusive resolution of the debate about the effect of religious priming on prosocial behavior – and about theoretically disputed effects more generally – requires a large-scale, preregistered replication project, which we consider to be the sole remedy for the adverse effects of experimenter bias and publication bias.

Neural and Temporal Dynamics Underlying Visual Selection for Action

The present study investigated the selection for action hypothesis, according to which a subjects action intention to perform a movement influences the way in which visual information is being processed. Subjects were instructed in separate blocks either to grasp or to point to a three-dimensional target-object and event-related potentials were recorded relative to stimulus onset. It was found that grasping compared with pointing resulted in a stronger N1 component and a subsequent selection negativity, which were localized to the lateral occipital complex. These effects suggest that the intention to grasp influences the processing of action-relevant features in ventral stream areas already at an early stage (e.g., enhanced processing of object orientation for grasping). These findings provide new insight in the neural and temporal dynamics underlying perception-action coupling and provide neural evidence for a selection for action principle in early human visual processing.

The N400-concreteness effect reflects the retrieval of semantic information during the preparation of meaningful actions

The present ERP study investigated when and how action semantics, i.e. the conceptual knowledge that we acquired over the course of our lives, is activated during the preparation of object-directed actions. Subjects were required to grasp one of two objects and were implicitly instructed to either perform a meaningful action (e.g. moving a cup towards the mouth) or a meaningless action with the object (e.g. moving a cup towards the eye). A larger anterior N400 was found for the preparation of meaningful compared to meaningless actions, likely reflecting the retrieval of action semantic information in case a meaningful action was required with the object. The distribution and the latency of the anterior N400-effect were strongly related to standard N400-repetition effects, thereby further corroborating the semantic nature of the effect. In sum, the present study provides new insight in the neural and temporal dynamics underlying semantics for action.

Neural evidence for impaired action selection in right hemiparetic cerebral palsy

Recent studies suggest that in addition to low-level motor impairments, individuals with hemiparetic cerebral palsy (HCP) are characterized by anticipatory action planning deficits as well. In the present EEG study we investigated the neural and temporal dynamics of action planning in participants with right-sided HCP (n=10) and in left-handed control subjects (n=10). An anticipatory planning task was used in which participants were required to grasp and rotate a hexagonal knob over different angles (60 degrees, 120 degrees or 180 degrees). At a behavioral level, participants with HCP were slower in their movements and often selected an inappropriate grip when grasping the object. At a neural level, individuals with HCP showed a strong reduction in the amplitude of the P2 component, likely reflecting an impaired process of action selection. In addition, a strong correlation was observed between the P2 amplitude and grasping and rotation times. The P2 component was localized to sources in the dorsal posterior cingulate cortex (dPCC), an area that is known to be involved in orienting visual body parts in space. Together these findings suggest that anticipatory planning deficits in cerebral palsy arise mainly due to an impaired process of action selection.

Pouring or chilling a bottle of wine: an fMRI study on the prospective planning of object-directed actions

This fMRI study investigates the neural mechanisms supporting the retrieval of action semantics. A novel motor imagery task was used in which participants were required to imagine planning actions with a familiar object (e.g. a toothbrush) or with an unfamiliar object (e.g. a pair of pliers) based on either goal-related information (i.e. where to move the object) or grip-related information (i.e. how to grasp the object). Planning actions with unfamiliar compared to familiar objects was slower and was associated with increased activation in the bilateral superior parietal lobe, the right inferior parietal lobe and the right insula. The stronger activation in parietal areas for unfamiliar objects fits well with the idea that parietal areas are involved in motor imagery and suggests that this process takes more effort in the case of novel or unfamiliar actions. In contrast, the planning of familiar actions resulted in increased activation in the anterior prefrontal cortex, suggesting that subjects maintained a stronger goal-representation when planning actions with familiar compared to unfamiliar objects. These findings provide further insight into the neural structures that support action semantic knowledge for the functional use of real-world objects and suggest that action semantic knowledge is activated most readily when actions are planned in a goal-directed manner.

Imitation of hand and tool actions is effector-independent

Following the theoretical notion that tools often extend one’s body, in the present study, we investigated whether imitation of hand or tool actions is modulated by effector-specific information. Subjects performed grasping actions toward an object with either a handheld tool or their right hand. Actions were initiated in response to pictures representing a grip at an object that could be congruent or incongruent with the required action (grip-type congruency). Importantly, actions could be cued by means of a tool cue, a hand cue, and a symbolic cue (effector-type congruency). For both hand and tool actions, an action congruency effect was observed, reflected in faster reaction times if the observed grip type was congruent with the required movement. However, neither hand actions nor tool actions were differentially affected by the effector represented in the picture (i.e., when performing a tool action, the action congruency effect was similar for tool cues and hand cues). This finding suggests that imitation of hand and tool actions is effector-independent and thereby supports generalist rather than specialist theories of imitation.

Learning to use novel objects: A training study on the acquisition of novel action representations

Many studies have suggested that the motor system is organized in a hierarchical fashion, around the prototypical end location associated with using objects. However, most studies supporting the hierarchical view have used well-known actions and objects that are highly over-learned. Accordingly, at present it is unclear if the hierarchical principle applies to learning the use of novel objects as well. In the present study we found that when learning to use a novel object subjects acquired an action representation of the end location associated with using the object, as evidenced by slower responses in an action observation task, when the object was presented at an incorrect end location. By showing the importance of knowledge about end locations when learning to use a novel object, the present study suggests that end locations are a fundamental organizing feature of the human motor system.