Jolien C. Francken

Embodied cognition: Taking the next step

Recent years have seen a large amount of empirical studies related to “embodied cognition.” While interesting and valuable, there is something dissatisfying with the current state of affairs in this research domain. Hypotheses tend to be underspecified, testing in general terms for embodied versus disembodied processing. The lack of specificity of current hypotheses can easily lead to an erosion of the embodiment concept, and result in a situation in which essentially any effect is taken as positive evidence. Such erosion is not helpful to the field and does not do justice to the importance of embodiment. Here we want to take stock, and formulate directions for how it can be studied in a more fruitful fashion. As an example we will describe few example studies that have investigated the role of sensori-motor systems in the coding of meaning (“embodied semantics”). Instead of focusing on the dichotomy between embodied and disembodied theories, we suggest that the field move forward and ask how and when sensori-motor systems and behavior are involved in cognition.

Immediate and long-term priming effects are independent of prime awareness

Subliminal primes are assumed to produce weaker and short-lived effects on subsequent behavior compared to clearly visible primes. However, this difference in priming effect may be due to differences in signal strength, rather than level of awareness. In the present study we manipulated prime discriminability by using metacontrast masks and pseudomasks, while keeping the prime strength equal. This manipulation resulted in large differences in discriminability of the primes. However, both immediate response priming and long-term response priming (measured with conflict adaptation) was equal for the poorly discriminable and well discriminable primes, and equal for groups that differed markedly in terms of how well they could discriminate the primes. Our findings imply that discriminability of information is independent of both the immediate and long-term effects that information can have on behavior.

The behavioral and neural effects of language on motion perception

Perception does not function as an isolated module but is tightly linked with other cognitive functions. Several studies have demonstrated an influence of language on motion perception, but it remains debated at which level of processing this modulation takes place. Some studies argue for an interaction in perceptual areas, but it is also possible that the interaction is mediated by “language areas” that integrate linguistic and visual information. Here, we investigated whether language–perception interactions were specific to the language-dominant left hemisphere by comparing the effects of language on visual material presented in the right (RVF) and left visual fields (LVF). Furthermore, we determined the neural locus of the interaction using fMRI. Participants performed a visual motion detection task. On each trial, the visual motion stimulus was presented in either the LVF or in the RVF, preceded by a centrally presented word (e.g., “rise”). The word could be congruent, incongruent, or neutral with regard to the direction of the visual motion stimulus that was presented subsequently. Participants were faster and more accurate when the direction implied by the motion word was congruent with the direction of the visual motion stimulus. Interestingly, the speed benefit was present only for motion stimuli that were presented in the RVF. We observed a neural counterpart of the behavioral facilitation effects in the left middle temporal gyrus, an area involved in semantic processing of verbal material. Together, our results suggest that semantic information about motion retrieved in language regions may automatically modulate perceptual decisions about motion.

Linguistic priors shape categorical perception

ABSTRACT This article reviews recent literature on the role of top-down feedback processes in semantic representations in the brain. Empirical studies on perception and theoretical models of semantic cognition show that sensory input is filtered and interpreted based on predictions from higher order cognitive areas. Here, we review the present evidence to the proposal that linguistic constructs, in particular, words, could serve as effective priors, facilitating perception and integration of sensory information. We address a number of theoretical questions arising from this assumption. The focus here is if linguistic categories have a direct top-down effect on early stages of perception; or rather interact with later processing stages such as semantic analysis. We discuss experimental approaches that could discriminate between these possibilities. Taken together, this article provides a review on the interaction between language and perception from the predictive perspective, and suggests avenues to investigate the underlying mechanisms from this perspective.

Exploring the automaticity of language-perception interactions: effects of attention and awareness

Previous studies have shown that language can modulate visual perception, by biasing and/or enhancing perceptual performance. However, it is still debated where in the brain visual and linguistic information are integrated, and whether the effects of language on perception are automatic and persist even in the absence of awareness of the linguistic material. Here, we aimed to explore the automaticity of language-perception interactions and the neural loci of these interactions in an fMRI study. Participants engaged in a visual motion discrimination task (upward or downward moving dots). Before each trial, a word prime was briefly presented that implied upward or downward motion (e.g., “rise”, “fall”). These word primes strongly influenced behavior: congruent motion words sped up reaction times and improved performance relative to incongruent motion words. Neural congruency effects were only observed in the left middle temporal gyrus, showing higher activity for congruent compared to incongruent conditions. This suggests that higher-level conceptual areas rather than sensory areas are the locus of language-perception interactions. When motion words were rendered unaware by means of masking, they still affected visual motion perception, suggesting that language-perception interactions may rely on automatic feed-forward integration of perceptual and semantic material in language areas of the brain.

Manipulating word awareness dissociates feed-forward from feedback models of language-perception interactions

Abstract Previous studies suggest that linguistic material can modulate visual perception, but it is unclear at which level of processing these interactions occur. Here we aim to dissociate between two competing models of language–perception interactions: a feed-forward and a feedback model. We capitalized on the fact that the models make different predictions on the role of feedback. We presented unmasked (aware) or masked (unaware) words implying motion (e.g. “rise,” “fall”), directly preceding an upward or downward visual motion stimulus. Crucially, masking leaves intact feed-forward information processing from low- to high-level regions, whereas it abolishes subsequent feedback. Under this condition, participants remained faster and more accurate when the direction implied by the motion word was congruent with the direction of the visual motion stimulus. This suggests that language–perception interactions are driven by the feed-forward convergence of linguistic and perceptual information at higher-level conceptual and decision stages.

Response to Desender & Van den Bussche: On the absence of a relationship between discriminability and priming

In their recent letter Desender and Van den Bussche (D&B) (2012) raise several issues with respect to our recent study (Francken, Gaal, et al., 2011), in which we describe a lack of relationship between how well a prime is perceived and its ability to influence behavior (i.e., priming). Here, we will shortly reply to these points. D&B argue that our results have no bearing on conscious vs. unconscious priming, since prime discriminability was above chance in both conditions. We fully agree with D&B that our study does not compare fully conscious with fully unconscious stimuli (as is also explicitly stated in the manuscript, where we speak about poorly discriminable and well discriminable primes). Nevertheless, there was a sizeable difference in discriminability between the low-visible and high-visible primes. If discriminability were relevant for priming, one would therefore expect the priming effects to differ between the conditions. Conversely, we found no differences in priming between low-visible and high-visible primes. Also, in spite of the large individual range of discriminability differences between metacontrast-masked and pseudomasked primes, there was no relationship between discriminability and priming effects, as visible from both a median-split analysis and correlation analysis (all p > 0.30). The second issue deals with the signal strength of the prime. According to D&B, signal strength was higher when primes were pseudomasked than metacontrast-masked, since they ‘‘critically differed in masking the direction of the prime’’. However, signal strength of the prime was defined in our study as the amount of priming it can elicit, not its visibility. Indeed, our study, as well as others (Vorberg, Mattler, et al., 2003) indicate that signal strength is dissociated from its visibility. Lastly, D&B describe their attempt to deal with the issues they raised, by ‘‘displaying the prime for an intermediate duration (40 ms), so that primes were unconscious on some trials and conscious on others’’. While we agree that this approach controls for potential stimulus differences, we would like to make a cautionary remark with respect to such a design. We believe that such a design may not allow for conclusive judgments about the role of awareness in priming per se. Namely, when not experimentally controlling awareness (e.g., by changing the masking parameters), but by relying on post hoc trial sorting, there are a number of (non-controlled) factors that may contribute to the effect. For example, it is well-known that there is an intimate relationship between awareness and attention (Dehaene & Changeux, 2011), which may well explain the larger priming effect for aware items in the study by D&B: on some trials, a subject’s attentional state was higher, leading to both stronger awareness and more priming. Also learning effects over time (subjects learn to ‘see’ the shape, therefore increasing awareness, while at the same time becoming more trained on the task, leading to larger prime congruency effects) may provide a partial explanation of the effect. In sum, we think that D&B raise some very important issues and we fully agree with D&B that ‘‘efforts should be made to compare conscious with truly unconscious conditions’’. The solution however, may not be to rely on post hoc sorting of data, but rather rigorous experimental control over the participant’s state of awareness.