Pia Tikka

Stimulus-related independent component and voxel-wise analysis of human brain activity during free viewing of a feature film.

Understanding how the brain processes stimuli in a rich natural environment is a fundamental goal of neuroscience. Here, we showed a feature film to 10 healthy volunteers during functional magnetic resonance imaging (fMRI) of hemodynamic brain activity. We then annotated auditory and visual features of the motion picture to inform analysis of the hemodynamic data. The annotations were fitted to both voxel-wise data and brain network time courses extracted by independent component analysis (ICA). Auditory annotations correlated with two independent components (IC) disclosing two functional networks, one responding to variety of auditory stimulation and another responding preferentially to speech but parts of the network also responding to non-verbal communication. Visual feature annotations correlated with four ICs delineating visual areas according to their sensitivity to different visual stimulus features. In comparison, a separate voxel-wise general linear model based analysis disclosed brain areas preferentially responding to sound energy, speech, music, visual contrast edges, body motion and hand motion which largely overlapped the results revealed by ICA. Differences between the results of IC- and voxel-based analyses demonstrate that thorough analysis of voxel time courses is important for understanding the activity of specific sub-areas of the functional networks, while ICA is a valuable tool for revealing novel information about functional connectivity which need not be explained by the predefined model. Our results encourage the use of naturalistic stimuli and tasks in cognitive neuroimaging to study how the brain processes stimuli in rich natural environments.

Functional Subdivision of Group-ICA Results of fMRI Data Collected during Cinema Viewing

Independent component analysis (ICA) can unravel functional brain networks from functional magnetic resonance imaging (fMRI) data. The number of the estimated components affects both the spatial pattern of the identified networks and their time-course estimates. Here group-ICA was applied at four dimensionalities (10, 20, 40, and 58 components) to fMRI data collected from 15 subjects who viewed a 15-min silent film (“At land” by Maya Deren). We focused on the dorsal attention network, the default-mode network, and the sensorimotor network. The lowest dimensionalities demonstrated most prominent activity within the dorsal attention network, combined with the visual areas, and in the default-mode network; the sensorimotor network only appeared with ICA comprising at least 20 components. The results suggest that even very low-dimensional ICA can unravel the most prominent functionally-connected brain networks. However, increasing the number of components gives a more detailed picture and functionally feasible subdivision of the major networks. These results improve our understanding of the hierarchical subdivision of brain networks during viewing of a movie that provides continuous stimulation embedded in an attention-directing narrative.

Listening to an Audio Drama Activates Two Processing Networks, One for All Sounds, Another Exclusively for Speech

Earlier studies have shown considerable intersubject synchronization of brain activity when subjects watch the same movie or listen to the same story. Here we investigated the across-subjects similarity of brain responses to speech and non-speech sounds in a continuous audio drama designed for blind people. Thirteen healthy adults listened for ∼19 min to the audio drama while their brain activity was measured with 3 T functional magnetic resonance imaging (fMRI). An intersubject-correlation (ISC) map, computed across the whole experiment to assess the stimulus-driven extrinsic brain network, indicated statistically significant ISC in temporal, frontal and parietal cortices, cingulate cortex, and amygdala. Group-level independent component (IC) analysis was used to parcel out the brain signals into functionally coupled networks, and the dependence of the ICs on external stimuli was tested by comparing them with the ISC map. This procedure revealed four extrinsic ICs of which two–covering non-overlapping areas of the auditory cortex–were modulated by both speech and non-speech sounds. The two other extrinsic ICs, one left-hemisphere-lateralized and the other right-hemisphere-lateralized, were speech-related and comprised the superior and middle temporal gyri, temporal poles, and the left angular and inferior orbital gyri. In areas of low ISC four ICs that were defined intrinsic fluctuated similarly as the time-courses of either the speech-sound-related or all-sounds-related extrinsic ICs. These ICs included the superior temporal gyrus, the anterior insula, and the frontal, parietal and midline occipital cortices. Taken together, substantial intersubject synchronization of cortical activity was observed in subjects listening to an audio drama, with results suggesting that speech is processed in two separate networks, one dedicated to the processing of speech sounds and the other to both speech and non-speech sounds.

Enactive cinema paves way for understanding complex real-time social interaction in neuroimaging experiments.

We outline general theoretical and practical implications of what we promote as enactive cinema for the neuroscientific study of online socio-emotional interaction. In a real-time functional magnetic resonance imaging (rt-fMRI) setting, participants are immersed in cinematic experiences that simulate social situations. While viewing, their physiological reactions—including brain responses—are tracked, representing implicit and unconscious experiences of the on-going social situations. These reactions, in turn, are analyzed in real-time and fed back to modify the cinematic sequences they are viewing while being scanned. Due to the engaging cinematic content, the proposed setting focuses on living-by in terms of shared psycho-physiological epiphenomena of experience rather than active coping in terms of goal-oriented motor actions. It constitutes a means to parametrically modify stimuli that depict social situations and their broader environmental contexts. As an alternative to studying the variation of brain responses as a function of a priori fixed stimuli, this method can be applied to survey the range of stimuli that evoke similar responses across participants at particular brain regions of interest.

Posterior Parietal Cortex Activity Reflects the Significance of Others' Actions During Natural Viewing

The posterior parietal cortex (PPC) has been associated with multiple stimulus‐driven (e.g., processing stimulus movements, providing visual signals for the motor system), goal‐directed (e.g., directing visual attention to a target, processing behavioral priority of intentions), and action‐related functions in previous studies with non‐naturalistic paradigms. Here, we examined how these functions reflect PPC activity during natural viewing. Fourteen healthy volunteers watched a re‐edited movie during functional magnetic resonance imaging (fMRI). Participants separately annotated behavioral priority (accounting for percepts, thoughts, and emotions) they had experienced during movie episodes. Movements in the movie were quantified with computer vision and eye movements were recorded from a separate group of subjects. Our results show that while overlapping dorsomedial PPC areas respond to episodes with multiple types of stimulus content, ventrolateral PPC areas exhibit enhanced activity when viewing goal‐directed human hand actions. Furthermore, PPC activity related to viewing goal‐directed human hand actions was more accurately explained by behavioral priority than by movements of the stimulus or eye movements. Taken together, our results suggest that PPC participates in perception of goal‐directed human hand actions, supporting the view that PPC has a special role in providing visual signals for the motor system (“how”), in addition to processing visual spatial movements (“where”). Hum Brain Mapp 35:4767–4776, 2014.

Haptic contents of a movie dynamically engage the spectator's sensorimotor cortex.

Observation of another persons actions and feelings activates brain areas that support similar functions in the observer, thereby facilitating inferences about the others mental and bodily states. In real life, events eliciting this kind of vicarious brain activations are intermingled with other complex, ever‐changing stimuli in the environment. One practical approach to study the neural underpinnings of real‐life vicarious perception is to image brain activity during movie viewing. Here the goal was to find out how observed haptic events in a silent movie would affect the spectators sensorimotor cortex. The functional state of the sensorimotor cortex was monitored by analyzing, in 16 healthy subjects, magnetoencephalographic (MEG) responses to tactile finger stimuli that were presented once per second throughout the session. Using canonical correlation analysis and spatial filtering, consistent single‐trial responses across subjects were uncovered, and their waveform changes throughout the movie were quantified. The long‐latency (85–175 ms) parts of the responses were modulated in concordance with the participants’ average moment‐by‐moment ratings of own engagement in the haptic content of the movie (correlation r = 0.49; ratings collected after the MEG session). The results, obtained by using novel signal‐analysis approaches, demonstrate that the functional state of the human sensorimotor cortex fluctuates in a fine‐grained manner even during passive observation of temporally varying haptic events. Hum Brain Mapp 37:4061–4068, 2016.

Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals

In the perceptual domain, it has been shown that the human brain is strongly shaped through experience, leading to expertise in highly-skilled professionals. What has remained unclear is whether specialization also shapes brain networks underlying mental imagery. In our fMRI study, we aimed to uncover modality-specific mental imagery specialization of film experts. Using multi-voxel pattern analysis we decoded from brain activity of professional cinematographers and sound designers whether they were imagining sounds or images of particular film clips. In each expert group distinct multi-voxel patterns, specific for the modality of their expertise, were found during classification of imagery modality. These patterns were mainly localized in the occipito-temporal and parietal cortex for cinematographers and in the auditory cortex for sound designers. We also found generalized patterns across perception and imagery that were distinct for the two expert groups: they involved frontal cortex for the cinematographers and temporal cortex for the sound designers. Notably, the mental representations of film clips and sounds of cinematographers contained information that went beyond modality-specificity. We were able to successfully decode the implicit presence of film genre from brain activity during mental imagery in cinematographers. The results extend existing neuroimaging literature on expertise into the domain of mental imagery and show that experience in visual versus auditory imagery can alter the representation of information in modality-specific association cortices.

Spatial variability of functional brain networks in early-blind and sighted subjects.

To further the understanding how the human brain adapts to early-onset blindness, we searched in early-blind and normally-sighted subjects for functional brain networks showing the most and least spatial variabilities across subjects. We hypothesized that the functional networks compensating for early-onset blindness undergo cortical reorganization. To determine whether reorganization of functional networks affects spatial variability, we used functional magnetic resonance imaging to compare brain networks, derived by independent component analysis, of 7 early-blind and 7 sighted subjects while they rested or listened to an audio drama. In both conditions, the blind compared with sighted subjects showed more spatial variability in a bilateral parietal network (comprising the inferior parietal and angular gyri and precuneus) and in a bilateral auditory network (comprising the superior temporal gyri). In contrast, a vision-related left-hemisphere-lateralized occipital network (comprising the superior, middle and inferior occipital gyri, fusiform and lingual gyri, and the calcarine sulcus) was less variable in blind than sighted subjects. Another visual network and a tactile network were spatially more variable in the blind than sighted subjects in one condition. We contemplate whether our results on inter-subject spatial variability of brain networks are related to experience-dependent brain plasticity, and we suggest that auditory and parietal networks undergo a stronger experience-dependent reorganization in the early-blind than sighted subjects while the opposite is true for the vision-related occipital network.

Model of Narrative Nowness for Neurocinematic Experiments

Cognitive neurosciences have made significant progress in learning about brain activity in situatedcognition, thanks to adopting stimuli that simulate immersion in naturalistic conditions insteadof ...

Enactive media – generalising from enactive cinema

This paper outlines general theoretical implications of enactive cinema for the broader framing of enactive media. The concept of enactive cinema has constituted a proposal for a novel kind of emotion-driven cinema genre, which emphasises unconscious interaction between the cinema spectator and the cinema. Instead of the spectator directly manipulating the narrative, the unfolding of the story is affected by the spectators emotional participation. The concept of enactive media refers to a systemic approach that may be applied to even other forms of narrative media besides cinema. It suggests going beyond the conventional and implicitly dualist concept of human–computer interaction and proposes a more holistic participation within a media system. The concept of second-order spectatorship, analogous, to that of the second-order authorship will be described.