Katja M. Mayer

The neural mechanisms of reliability weighted integration of shape information from vision and touch

Behaviourally, humans have been shown to integrate multisensory information in a statistically-optimal fashion by averaging the individual unisensory estimates according to their relative reliabilities. This form of integration is optimal in that it yields the most reliable (i.e. least variable) multisensory percept. The present study investigates the neural mechanisms underlying integration of visual and tactile shape information at the macroscopic scale of the regional BOLD response. Observers discriminated the shapes of ellipses that were presented bimodally (visual-tactile) or visually alone. A 2 × 5 factorial design manipulated (i) the presence vs. absence of tactile shape information and (ii) the reliability of the visual shape information (five levels). We then investigated whether regional activations underlying tactile shape discrimination depended on the reliability of visual shape. Indeed, in primary somatosensory cortices (bilateral BA2) and the superior parietal lobe the responses to tactile shape input were increased when the reliability of visual shape information was reduced. Conversely, tactile inputs suppressed visual activations in the right posterior fusiform gyrus, when the visual signal was blurred and unreliable. Somatosensory and visual cortices may sustain integration of visual and tactile shape information either via direct connections from visual areas or top-down effects from higher order parietal areas.

TBSS and probabilistic tractography reveal white matter connections for attention to object features

Abstract Selective attention to features of interest facilitates object processing in a cluttered and dynamic environment. Previous research found that distinct networks of regions across cortex are activated depending on the attended feature. These networks typically consist of posterior feature-preferring regions and anterior regions involved in attentional processes. In the current study, we investigated the role of white matter connections between the posterior and anterior regions within these networks for attention to features of novel colored dynamic objects. We asked participants to perform a 1-back feature-attention task while we acquired both functional and diffusion-weighted images. Using tract-based spatial statistics and probabilistic tractography, we found that the right superior longitudinal fasciculus (SLF) connected posterior and anterior object-processing regions and that voxels within the SLF correlated with response times on the task. Posterior and anterior regions that were anatomically connected also had increased functional connectivity relative to posterior and anterior regions that were not connected. Our results demonstrate that both functional and structural information has to be taken into account to understand selective attention and object perception.

Duration perception in crossmodally-defined intervals

How humans perform duration judgments with multisensory stimuli is an ongoing debate. Here, we investigated how sub-second duration judgments are achieved by asking participants to compare the duration of a continuous sound to the duration of an empty interval in which onset and offset were marked by signals of different modalities using all combinations of visual, auditory and tactile stimuli. The pattern of perceived durations across five stimulus durations (ranging from 100 ms to 900 ms) follows the Vierordt Law. Furthermore, intervals with a sound as onset (audio-visual, audio-tactile) are perceived longer than intervals with a sound as offset. No modality ordering effect is found for visualtactile intervals. To infer whether a single modality-independent or multiple modality-dependent time-keeping mechanisms exist we tested whether perceived duration follows a summative or a multiplicative distortion pattern by fitting a model to all modality combinations and durations. The results confirm that perceived duration depends on sensory latency (summative distortion). Instead, we did not find evidence for multiplicative distortions. The results of the model and the behavioural data support the concept of a single time-keeping mechanism that allows for judgments of durations marked by multisensory stimuli.

Do People “Pop Out”?

The human body is a highly familiar and socially very important object. Does this mean that the human body has a special status with respect to visual attention? In the current paper we tested whether people in natural scenes attract attention and “pop out” or, alternatively, are at least searched for more efficiently than targets of another category (machines). Observers in our study searched a visual array for dynamic or static scenes containing humans amidst scenes containing machines and vice versa. The arrays consisted of 2, 4, 6 or 8 scenes arranged in a circular array, with targets being present or absent. Search times increased with set size for dynamic and static human and machine targets, arguing against pop out. However, search for human targets was more efficient than for machine targets as indicated by shallower search slopes for human targets. Eye tracking further revealed that observers made more first fixations to human than to machine targets and that their on-target fixation durations were shorter for human compared to machine targets. In summary, our results suggest that searching for people in natural scenes is more efficient than searching for other categories even though people do not pop out.

Automatic processing of unattended object features by functional connectivity.

Observers can selectively attend to object features that are relevant for a task. However, unattended task-irrelevant features may still be processed and possibly integrated with the attended features. This study investigated the neural mechanisms for processing both task-relevant (attended) and task-irrelevant (unattended) object features. The Garner paradigm was adapted for functional magnetic resonance imaging (fMRI) to test whether specific brain areas process the conjunction of features or whether multiple interacting areas are involved in this form of feature integration. Observers attended to shape, color, or non-rigid motion of novel objects while unattended features changed from trial to trial (change blocks) or remained constant (no-change blocks) during a given block. This block manipulation allowed us to measure the extent to which unattended features affected neural responses which would reflect the extent to which multiple object features are automatically processed. We did not find Garner interference at the behavioral level. However, we designed the experiment to equate performance across block types so that any fMRI results could not be due solely to differences in task difficulty between change and no-change blocks. Attention to specific features localized several areas known to be involved in object processing. No area showed larger responses on change blocks compared to no-change blocks. However, psychophysiological interaction (PPI) analyses revealed that several functionally-localized areas showed significant positive interactions with areas in occipito-temporal and frontal areas that depended on block type. Overall, these findings suggest that both regional responses and functional connectivity are crucial for processing multi-featured objects.

Recently learned foreign abstract and concrete nouns are represented in distinct cortical networks similar to the native language

In the native language, abstract and concrete nouns are represented in distinct areas of the cerebral cortex. Currently, it is unknown whether this is also the case for abstract and concrete nouns of a foreign language. Here, we taught adult native speakers of German 45 abstract and 45 concrete nouns of a foreign language. After learning the nouns for 5 days, participants performed a vocabulary translation task during functional magnetic resonance imaging. Translating abstract nouns in contrast to concrete nouns elicited responses in regions that are also responsive to abstract nouns in the native language: the left inferior frontal gyrus and the left middle and superior temporal gyri. Concrete nouns elicited larger responses in the angular gyri bilaterally and the left parahippocampal gyrus than abstract nouns. The cluster in the left angular gyrus showed psychophysiological interaction (PPI) with the left lingual gyrus. The left parahippocampal gyrus showed PPI with the posterior cingulate cortex. Similar regions have been previously found for concrete nouns in the native language. The results reveal similarities in the cortical representation of foreign language nouns with the representation of native language nouns that already occur after 5 days of vocabulary learning. Furthermore, we showed that verbal and enriched learning methods were equally suitable to teach foreign abstract and concrete nouns. Hum Brain Mapp 38:4398–4412, 2017.