Jasper J. F. van den Bosch

Cross-cultural differences in crossmodal correspondences between basic tastes and visual features

We report a cross-cultural study designed to investigate crossmodal correspondences between a variety of visual features (11 colors, 15 shapes, and 2 textures) and the five basic taste terms (bitter, salty, sour, sweet, and umami). A total of 452 participants from China, India, Malaysia, and the USA viewed color patches, shapes, and textures online and had to choose the taste term that best matched the image and then rate their confidence in their choice. Across the four groups of participants, the results revealed a number of crossmodal correspondences between certain colors/shapes and bitter, sour, and sweet tastes. Crossmodal correspondences were also documented between the color white and smooth/rough textures on the one hand and the salt taste on the other. Cross-cultural differences were observed in the correspondences between certain colors, shapes, and one of the textures and the taste terms. The taste-patterns shown by the participants from the four countries tested in the present study are quite different from one another, and these differences cannot easily be attributed merely to whether a country is Eastern or Western. These findings therefore highlight the impact of cultural background on crossmodal correspondences. As such, they raise a number of interesting questions regarding the neural mechanisms underlying crossmodal correspondences.

Cross-cultural color-odor associations

Colors and odors are associated; for instance, people typically match the smell of strawberries to the color pink or red. These associations are forms of crossmodal correspondences. Recently, there has been discussion about the extent to which these correspondences arise for structural reasons (i.e., an inherent mapping between color and odor), statistical reasons (i.e., covariance in experience), and/or semantically-mediated reasons (i.e., stemming from language). The present study probed this question by testing color-odor correspondences in 6 different cultural groups (Dutch, Netherlands-residing-Chinese, German, Malay, Malaysian-Chinese, and US residents), using the same set of 14 odors and asking participants to make congruent and incongruent color choices for each odor. We found consistent patterns in color choices for each odor within each culture, showing that participants were making non-random color-odor matches. We used representational dissimilarity analysis to probe for variations in the patterns of color-odor associations across cultures; we found that US and German participants had the most similar patterns of associations, followed by German and Malay participants. The largest group differences were between Malay and Netherlands-resident Chinese participants and between Dutch and Malaysian-Chinese participants. We conclude that culture plays a role in color-odor crossmodal associations, which likely arise, at least in part, through experience.

FOSE: a framework for open science evaluation.

Pre-publication peer review of scientific literature in its present state suffers from a lack of evaluation validity and transparency to the community. Inspired by social networks, we propose a framework for the open exchange of post-publication evaluation to complement the current system. We first formulate a number of necessary conditions that should be met by any design dedicated to perform open scientific evaluation. To introduce our framework, we provide a basic data standard and communication protocol. We argue for the superiority of a provider-independent framework, over a few isolated implementations, which allows the collection and analysis of open evaluation content across a wide range of diverse providers like scientific journals, research institutions, social networks, publishers websites, and more. Furthermore, we describe how its technical implementation can be achieved by using existing web standards and technology. Finally, we illustrate this with a set of examples and discuss further potential.

Neuroimaging of the bilingual brain: Structural brain correlates of listening and speaking in a second language

Diffusion tensor imaging was used to compare white matter structure between American monolingual and Spanish-English bilingual adults living in the United States. In the bilingual group, relationships between white matter structure and naturalistic immersive experience in listening to and speaking English were additionally explored. White matter structural differences between groups were found to be bilateral and widespread. In the bilingual group, experience in listening to English was more robustly correlated with decreases in radial and mean diffusivity in anterior white matter regions of the left hemisphere, whereas experience in speaking English was more robustly correlated with increases in fractional anisotropy in more posterior left hemisphere white matter regions. The findings suggest that (a) foreign language immersion induces neuroplasticity in the adult brain, (b) the degree of alteration is proportional to language experience, and (c) the modes of immersive language experience have more robust effects on different brain regions and on different structural features.

Touching Sounds: Thalamocortical Plasticity and the Neural Basis of Multisensory Integration

To date, noninvasive neuroimaging research on multisensory perception has focused on cortical activations. In a series of elegant functional magnetic resonance imaging experiments, Beauchamp and Ro recently investigated altered cortical activations associated with acquired sound-touch synesthesia resulting from a thalamic lesion. Their findings highlight the important role of intact thalamocortical projections for preventing illusory crossmodal perception and for underlying reliable multisensory integration.

Synaesthesia or Vivid Imagery? A Single Case fMRI Study of Visually Induced Olfactory Perception

The most common form of synaesthesia is grapheme-colour synaesthesia. However, rarer forms of synaesthesia also exist, such as word-gustatory and olfactory-gustatory synaesthesia, whereby a word or smell will induce a specific. In this study we describe a single individual (LJ) who experiences a concurrent olfactory stimulus when presented with congruent visual images. For some visual stimuli, he perceives a strong and automatic olfactory percept, which has existed throughout his life. In this study, we explore whether his experiences are a new form of synaesthesia or simply vivid imagery. Unlike other forms of synaesthesia, the concurrent odour is congruent to the visual inducer. For example, a photograph of dress shoes will elicit the smell of leather. We presented LJ and several control participants with 75 images of everyday objects. Their task was to indicate the strength of any perceived odours induced by the visual images. LJ rated several of the images as inducing a concurrent odour, while controls did not have any such percept. Images that LJ reported as inducing the strongest odours were used, along with colour-matched control images, in the context of an fMRI experiment. Participants were given a one-back task to maintain attention. A block-design odour localizer was presented to localize the piriform cortex (primary olfactory cortex). We found an increased BOLD response in the piriform cortex for the odour-inducing images compared to the control images in LJ. There was no difference in BOLD response between these two stimulus types in the control participants. A subsequent olfactory imagery task did not elicit enhanced activity in the piriform cortex in LJ, suggesting his perceptual experiences may not be based on olfactory imagery.

Multisensory Functional Magnetic Resonance Imaging

Since its invention almost two decades ago functional magnetic resonance imaging (fMRI) has become the prime research methodology in human neuroscience. Its capabilities continue to evolve based on combined improvements of scanner hardware, experimental designs, and data analysis tools. Within the rapidly growing field of multisensory research the use of noninvasive neuroimaging techniques in general and fMRI in particular is also of increasing relevance. For several years, discussion in the multisensory fMRI community has mainly focused on principles of and statistical criteria for multisensory integration. The recent availability of more sophisticated experimental designs and increasingly sensitive (multivariate) analysis tools allows multisensory researchers to (noninvasively) differentiate between regional and neuronal convergence and to reveal the connectional basis of human multisensory integration.

Mind the drift - improving sensitivity to fMRI pattern information by accounting for temporal pattern drift

Analyzing functional magnetic resonance imaging (fMRI) pattern similarity is becoming increasingly popular because it allows one to relate distributed patterns of voxel activity to continuous perceptual and cognitive states of the human brain. Here we show that fMRI pattern similarity estimates are severely affected by temporal pattern drifts in fMRI data – even after voxel-wise detrending. For this particular dataset, the drift effect obscures orientation information as measured by fMRI pattern dissimilarities. We demonstrate that orientation information can be recovered using three different methods: 1. Regressing out the drift component through linear modeling; 2. Computing representational distances between conditions measured in independent imaging runs; 3. Crossvalidation of pattern distance estimates. One possible source of temporal pattern drift could be random walk like fluctuations — physiological or scanner related — occurring within single voxel timecourses. This explanation is consistent with voxel-wise detrending not alleviating pattern drift effects. In addition, this would explain why crossvalidated pattern distances are robust to temporal drift because a random walk process is expected to give rise to non-replicable drift directions. Given these findings, we recommend that future fMRI studies take pattern drift into account when analyzing pattern similarity as this can greatly enhance the sensitivity to experimental effects of interest.

The influence of gender incongruence on the McGurk-percept: A combined behavioural and fMRI study

The McGurk-effect (McGurk and MacDonald, 1976) is a robust illusion which is broadly studied in the context of audiovisual integration. In the illusion, auditory speech perception is modified by discrepant visual lip-movements when presented synchronously, leading to a third, not physically present percept. There is an ongoing debate as to whether the percept is integrated early in the sensory system or if it is susceptible to cognitive intervention. Here, the McGurk-effect was studied using gender congruency (face/voice stimuli) as the cognitive intervention to address the question of integration. We first investigated changes in prevalence and reaction times due to perceived gender discrepancy during the McGurk-percept. In the second experiment, neural correlates of the gender discrepant McGurk-percept were studied using fMRI. We did not find differences in reaction times or accuracy between gender congruent or incongruent stimuli. Thus it appears the McGurk-percept itself is unaffected by gender incongruent stimuli on a behavioural level. In the fMRI experiment, we found the superior temporal gyrus to reveal a significantly increased activity when the McGurk-effect is perceived as compared to non-McGurk trials, but no effect in this area was found for gender congruency. However, a 2 × 2 whole brain ANOVA revealed a significant interaction in the face identity processing area (fusiform gyrus) as well as inferior parietal gyrus and superior colliculus. We suggest a mechanism, allowing a stable illusory percept in the gender discrepant McGurk-percept to be facilitated. To our knowledge this is the first study to demonstrate on a neural level that gender congruency affects processing of the McGurk-effect.

Neurocognitive Development of the Resolution of Selective Visuo-Spatial Attention: Functional MRI Evidence From Object Tracking

Our ability to select relevant information from the environment is limited by the resolution of attention – i.e., the minimum size of the region that can be selected. Neural mechanisms that underlie this limit and its development are not yet understood. Functional magnetic resonance imaging (fMRI) was performed during an object tracking task in 7- and 11-year-old children, and in young adults. Object tracking activated canonical fronto-parietal attention systems and motion-sensitive area MT in children as young as 7 years. Object tracking performance improved with age, together with stronger recruitment of parietal attention areas and a shift from low-level to higher-level visual areas. Increasing the required resolution of spatial attention – which was implemented by varying the distance between target and distractors in the object tracking task – led to activation increases in fronto-insular cortex, medial frontal cortex including anterior cingulate cortex (ACC) and supplementary motor area, superior colliculi, and thalamus. This core circuitry for attentional precision was recruited by all age groups, but ACC showed an age-related activation reduction. Our results suggest that age-related improvements in selective visual attention and in the resolution of attention are characterized by an increased use of more functionally specialized brain regions during the course of development.