Dominique Goltz

Multivariate Machine Learning Methods for Fusing Multimodal Functional Neuroimaging Data

Multimodal data are ubiquitous in engineering, communications, robotics, computer vision, or more generally speaking in industry and the sciences. All disciplines have developed their respective sets of analytic tools to fuse the information that is available in all measured modalities. In this paper, we provide a review of classical as well as recent machine learning methods (specifically factor models) for fusing information from functional neuroimaging techniques such as: LFP, EEG, MEG, fNIRS, and fMRI. Early and late fusion scenarios are distinguished, and appropriate factor models for the respective scenarios are presented along with example applications from selected multimodal neuroimaging studies. Further emphasis is given to the interpretability of the resulting model parameters, in particular by highlighting how factor models relate to physical models needed for source localization. The methods we discuss allow for the extraction of information from neural data, which ultimately contributes to 1) better neuroscientific understanding; 2) enhance diagnostic performance; and 3) discover neural signals of interest that correlate maximally with a given cognitive paradigm. While we clearly study the multimodal functional neuroimaging challenge, the discussed machine learning techniques have a wide applicability, i.e., in general data fusion, and may thus be informative to the general interested reader.

Functional connectivity-based parcellation of the human sensorimotor cortex

Task‐based functional magnetic resonance imaging (fMRI) has been successfully employed to obtain somatotopic maps of the human sensorimotor cortex. Here, we showed through direct comparison that a similar functional map can be obtained, independently of a task, by performing a connectivity‐based parcellation of the sensorimotor cortex based on resting‐state fMRI. Cortex corresponding to two adjacent Brodmann areas (BA 3 and BA 4) was selected as the sensorimotor area. Parcellation was obtained along a medial–lateral axis, which was confirmed to be somatotopic (corresponding roughly to an upper, middle and lower limb, respectively) by comparing it with maps obtained using motoric task‐based fMRI in the same participants. Interestingly, the resting‐state parcellation map demonstrated higher correspondence to the task‐based divisions after individuals performed the motor task. Using the resting‐state fMRI data, we also observed higher functional correlations between the centrally located hand region and the other two regions, than between the foot and tongue. The functional relevance of these somatosensory parcellation results indicates the feasibility of a wide range of potential applications to brain mapping.

Maintenance and manipulation of somatosensory information in ventrolateral prefrontal cortex.

Neuroimaging studies of working memory (WM) suggest that prefrontal cortex may assist sustained maintenance, but also internal manipulation, of stimulus representations in lower‐level areas. A different line of research in the somatosensory domain indicates that neuronal activity in ventrolateral prefrontal cortex (VLPFC) may also represent specific memory contents in itself, however leaving open to what extent top‐down control on lower‐level areas is exerted, or how internal manipulation processes are implemented. We used functional imaging and connectivity analysis to study static maintenance and internal manipulation of tactile working memory contents after physically identical stimulation conditions, in human subjects. While both tasks recruited similar subareas in the inferior frontal gyrus (IFG) in VLPFC, static maintenance of the tactile information was additionally characterized by increased functional coupling between IFG and primary somatosensory cortex. Independently, during internal manipulation, a quantitative representation of the task‐relevant information was evident in IFG itself, even in the absence of physical stimulation. Together, these findings demonstrate the functional diversity of activity within VLPFC according to different working memory demands, and underline the role of IFG as a core region in sensory WM processing. Hum Brain Mapp 35:2412–2423, 2014.

Sustained spatial attention to vibrotactile stimulation in the flutter range: relevant brain regions and their interaction.

The present functional magnetic resonance imaging (fMRI) study was designed to get a better understanding of the brain regions involved in sustained spatial attention to tactile events and to ascertain to what extent their activation was correlated. We presented continuous 20 Hz vibrotactile stimuli (range of flutter) concurrently to the left and right index fingers of healthy human volunteers. An arrow cue instructed subjects in a trial-by-trial fashion to attend to the left or right index finger and to detect rare target events that were embedded in the vibrotactile stimulation streams. We found blood oxygen level-dependent (BOLD) attentional modulation in primary somatosensory cortex (SI), mainly covering Brodmann area 1, 2, and 3b, as well as in secondary somatosensory cortex (SII), contralateral to the to-be-attended hand. Furthermore, attention to the right (dominant) hand resulted in additional BOLD modulation in left posterior insula. All of the effects were caused by an increased activation when attention was paid to the contralateral hand, except for the effects in left SI and insula. In left SI, the effect was related to a mixture of both a slight increase in activation when attention was paid to the contralateral hand as well as a slight decrease in activation when attention was paid to the ipsilateral hand (i.e., the tactile distraction condition). In contrast, the effect in left posterior insula was exclusively driven by a relative decrease in activation in the tactile distraction condition, which points to an active inhibition when tactile information is irrelevant. Finally, correlation analyses indicate a linear relationship between attention effects in intrahemispheric somatosensory cortices, since attentional modulation in SI and SII were interrelated within one hemisphere but not across hemispheres. All in all, our results provide a basis for future research on sustained attention to continuous vibrotactile stimulation in the range of flutter.

Impairing somatosensory working memory using rTMS.

Numerous studies in animals and humans have related central aspects of somatosensory working memory function to neural activity in the inferior frontal gyrus (IFG). However, as previous studies have almost exclusively used correlational analyses, the question whether sustained neural activity in the IFG is causally involved in successful maintenance of somatosensory information remains unanswered. We used an online repetitive transcranial magnetic stimulation (rTMS) protocol to disrupt neuronal activity in the IFG while participants were maintaining tactile information throughout the delay for later comparison against a probe stimulus. rTMS impaired participants’ performance in the working memory task, but not in a physically matched perceptual control task. Targeting the IFG in either hemisphere led to comparable working memory impairment. Our results show that the neural activity in the IFG plays a causal role in successful maintenance of somatosensory information.

Connections between Intraparietal Sulcus and a Sensorimotor Network Underpin Sustained Tactile Attention

Previous studies on sustained tactile attention draw conclusions about underlying cortical networks by averaging over experimental conditions without considering attentional variance in single trials. This may have formed an imprecise picture of brain processes underpinning sustained tactile attention. In the present study, we simultaneously recorded EEG-fMRI and used modulations of steady-state somatosensory evoked potentials (SSSEPs) as a measure of attentional trial-by-trial variability. Therefore, frequency-tagged streams of vibrotactile stimulations were simultaneously presented to both index fingers. Human participants were cued to sustain attention to either the left or right finger stimulation and to press a button whenever they perceived a target pulse embedded in the to-be-attended stream. In-line with previous studies, a classical general linear model (GLM) analysis based on cued attention conditions revealed increased activity mainly in somatosensory and cerebellar regions. Yet, parametric modeling of the BOLD response using simultaneously recorded SSSEPs as a marker of attentional trial-by-trial variability quarried the intraparietal sulcus (IPS). The IPS in turn showed enhanced functional connectivity to a modality-unspecific attention network. However, this was only revealed on the basis of cued attention conditions in the classical GLM. By considering attentional variability as captured by SSSEPs, the IPS showed increased connectivity to a sensorimotor network, underpinning attentional selection processes between competing tactile stimuli and action choices (press a button or not). Thus, the current findings highlight the potential value by considering attentional variations in single trials and extend previous knowledge on the role of the IPS in tactile attention.

P 186. Inter-and intra-individual differences in the attentional modulation of the somatosensory steady state signal

Introduction In electroencephalography (EEG) research, frequency-tagging has become an important measure for investigating sustained attention in the field of vision, audition and both (cf. Keitel et al., 2011; Saupe et al., 2009; Muller et al., 2003). Frequency-tagged stimuli elicit the steady state evoked potential (SSEP), which is an oscillatory brain response of the same frequency as the driving frequency. Crucially, paying attention to such a stimulus causes an increase in SSEP amplitude compared to when the respective stimulus had to be ignored. In the somatosensory domain however, attentional modulation of the steady state signal seems to be highly variable across and within subjects; i.e. there is either an increased or decreased amplitude when attention is paid to a vibrotactile stimulus. Interestingly, despite this opposing pattern behavioral performance showed that participants seemed to perform the task correctly. The present study wanted to shed light on possible factors causing these inter-and intra-individual differences. Methods and hypotheses We conducted a simultaneous EEG and functional magnetic resonance imaging (fMRI) study because both methods provide us with different markers of attention and brain states at different time scales. We expected to find a direct relationship between the attentional modulation seen in the EEG and fMRI signal. Participants with an increased amplitude of the EEG steady state signal should show an increased BOLD response in primary somatosensory cortex compared to participants with a decreased amplitude. Besides differences in pre-stimulus EEG oscillations (e.g. μ-alpha or μ-beta) and resting state fMRI measures might account for inter and intra-individual alterations in attention effects. Conclusion The present study allows the direct comparisons of EEG, fMRI and behavioral attention markers and thereby paves the road for multi-method approaches in sustained somatosensory attention research.