Leonhard Schilbach

Toward a second-person neuroscience

In spite of the remarkable progress made in the burgeoning field of social neuroscience, the neural mechanisms that underlie social encounters are only beginning to be studied and couldxa0-xa0paradoxicallyxa0-xa0be seen as representing the dark matter of social neuroscience. Recent conceptual and empirical developments consistently indicate the need for investigations that allow the study of real-time social encounters in a truly interactive manner. This suggestion is based on the premise that social cognition is fundamentally different when we are in interaction with others rather than merely observing them. In this article, we outline the theoretical conception of a second-person approach to other minds and review evidence from neuroimaging, psychophysiological studies, and related fields to argue for the development of a second-person neuroscience, which will help neuroscience to really go social; this may also be relevant for our understanding of psychiatric disorders construed as disorders of social cognition.

Being with virtual others: Neural correlates of social interaction

To characterize the neural correlates of being personally involved in social interaction as opposed to being a passive observer of social interaction between others we performed an fMRI study in which participants were gazed at by virtual characters (ME) or observed them looking at someone else (OTHER). In dynamic animations virtual characters then showed socially relevant facial expressions as they would appear in greeting and approach situations (SOC) or arbitrary facial movements (ARB). Differential neural activity associated with ME>OTHER was located in anterior medial prefrontal cortex in contrast to the precuneus for OTHER>ME. Perception of socially relevant facial expressions (SOC>ARB) led to differentially increased neural activity in ventral medial prefrontal cortex. Perception of arbitrary facial movements (ARB>SOC) differentially activated the middle temporal gyrus. The results, thus, show that activation of medial prefrontal cortex underlies both the perception of social communication indicated by facial expressions and the feeling of personal involvement indicated by eye gaze. Our data also demonstrate that distinct regions of medial prefrontal cortex contribute differentially to social cognition: whereas the ventral medial prefrontal cortex is recruited during the analysis of social content as accessible in interactionally relevant mimic gestures, differential activation of a more dorsal part of medial prefrontal cortex subserves the detection of self-relevance and may thus establish an intersubjective context in which communicative signals are evaluated.

Minds made for sharing: Initiating joint attention recruits reward-related neurocircuitry

The ability and motivation to share attention is a unique aspect of human cognition. Despite its significance, the neural basis remains elusive. To investigate the neural correlates of joint attention, we developed a novel, interactive research paradigm in which participants gaze behavior—as measured by an eye tracking device—was used to contingently control the gaze of a computer-animated character. Instructed that the character on screen was controlled by a real person outside the scanner, 21 participants interacted with the virtual other while undergoing fMRI. Experimental variations focused on leading versus following the gaze of the character when fixating one of three objects also shown on the screen. In concordance with our hypotheses, results demonstrate, firstly, that following someone elses gaze to engage in joint attention resulted in activation of anterior portion of medial prefrontal cortex (MPFC) known to be involved in the supramodal coordination of perceptual and cognitive processes. Secondly, directing someone elses gaze toward an object activated the ventral striatum which—in light of ratings obtained from participants—appears to underlie the hedonic aspects of sharing attention. The data, therefore, support the idea that other-initiated joint attention relies upon recruitment of MPFC previously related to the “meeting of minds.” In contrast, self-initiated joint attention leads to a differential increase of neural activity in reward-related brain areas, which might contribute to the uniquely human motivation to engage in the sharing of experiences.

Parsing the neural correlates of moral cognition: ALE meta-analysis on morality, theory of mind, and empathy

Morally judicious behavior forms the fabric of human sociality. Here, we sought to investigate neural activity associated with different facets of moral thought. Previous research suggests that the cognitive and emotional sources of moral decisions might be closely related to theory of mind, an abstract-cognitive skill, and empathy, a rapid-emotional skill. That is, moral decisions are thought to crucially refer to other persons’ representation of intentions and behavioral outcomes as well as (vicariously experienced) emotional states. We thus hypothesized that moral decisions might be implemented in brain areas engaged in ‘theory of mind’ and empathy. This assumption was tested by conducting a large-scale activation likelihood estimation (ALE) meta-analysis of neuroimaging studies, which assessed 2,607 peak coordinates from 247 experiments in 1,790 participants. The brain areas that were consistently involved in moral decisions showed more convergence with the ALE analysis targeting theory of mind versus empathy. More specifically, the neurotopographical overlap between morality and empathy disfavors a role of affective sharing during moral decisions. Ultimately, our results provide evidence that the neural network underlying moral decisions is probably domain-global and might be dissociable into cognitive and affective sub-systems.

It’s in your eyes—using gaze-contingent stimuli to create truly interactive paradigms for social cognitive and affective neuroscience

The field of social neuroscience has made remarkable progress in elucidating the neural mechanisms of social cognition. More recently, the need for new experimental approaches has been highlighted that allow studying social encounters in a truly interactive manner by establishing online reciprocity in social interaction. In this article, we present a newly developed adaptation of a method which uses eyetracking data obtained from participants in real time to control visual stimulation during functional magnetic resonance imaging, thus, providing an innovative tool to generate gaze-contingent stimuli in spite of the constraints of this experimental setting. We review results of two paradigms employing this technique and demonstrate how gaze data can be used to animate a virtual character whose behavior becomes responsive to being looked at allowing the participant to engage in online interaction with this virtual other in real-time. Possible applications of this setup are discussed highlighting the potential of this development as a new tool of the trade in social cognitive and affective neuroscience.

Definition and characterization of an extended social-affective default network

Recent evidence suggests considerable overlap between the default mode network (DMN) and regions involved in social, affective and introspective processes. We considered these overlapping regions as the social-affective part of the DMN. In this study, we established a robust mapping of the underlying brain network formed by these regions and those strongly connected to them (the extended social-affective default network). We first seeded meta-analytic connectivity modeling and resting-state analyses in the meta-analytically defined DMN regions that showed statistical overlap with regions associated with social and affective processing. Consensus connectivity of each seed was subsequently delineated by a conjunction across both connectivity analyses. We then functionally characterized the ensuing regions and performed several cluster analyses. Among the identified regions, the amygdala/hippocampus formed a cluster associated with emotional processes and memory functions. The ventral striatum, anterior cingulum, subgenual cingulum and ventromedial prefrontal cortex formed a heterogeneous subgroup associated with motivation, reward and cognitive modulation of affect. Posterior cingulum/precuneus and dorsomedial prefrontal cortex were associated with mentalizing, self-reference and autobiographic information. The cluster formed by the temporo-parietal junction and anterior middle temporal sulcus/gyrus was associated with language and social cognition. Taken together, the current work highlights a robustly interconnected network that may be central to introspective, socio-affective, that is, self- and other-related mental processes.

A second-person approach to other minds

In a recent Review (The functional role of the parieto-frontal mirror circuit: interpretations and misinterpretations. Nature Rev. Neurosci. 11, 264–274 (2010))1, Rizzolatti and Sinigaglia examine the role of the parieto-frontal cortex in action observation and action execution in monkeys and humans. They suggest that its functional relevance for cognition results from a ‘mirror mechanism’ that allows individuals to understand the actions of another individual by giving the observer a ‘first-person grasp’ of the motor goals and intentions of the other. Although the Review seems to be well balanced by considering many of the recent arguments put forward against this simulationist interpretation of the evidence and by recognizing the relevance of other mechanisms2, it fails to make reference to the enactive account of cognition, which stresses that cognition is achieved by an animal’s active exploration of and coupling with its environment3. This seems to be most relevant, as an extension of this account to the social domain suggests that social cognition is fundamentally different when an individual is actively and directly interacting with others. In such cases, an individual adopts a ‘second-person perspective’ in which interaction with the other can be thought of as essential or even constitutive for social cognition, rather than merely observing others and relying on a ‘first(or third-) person grasp’ of their mental states4–6. Apart from this being a conceptual ‘blind spot’ of the Review, the distinction of being directly engaged in a real-time interaction with someone else versus merely observing others also seems to be highly relevant to the discussion of the empirical evidence. The authors make reference to functional neuroimaging data that implicate two large-scale neural networks in understanding conspecifics’ actions and intentions (the ‘mirror neuron system’ and the ‘mentalizing network’) and suggest that “there are currently no neurophysiological data that can explain how the ‘mentalizing network’ might work”. This may be so, but it could also be argued that the existence of seemingly disparate sets of data is due to the inability of social neuroscience — paradoxically, but largely due to methodological constraints — to investigate real-time interactions between individuals in an ecologically valid way5. Therefore, it is unclear how activity in the parieto-frontal cortex and the mentalizing network during action observation may be modulated by the degree to which human observers perceive themselves as participants of an ongoing interaction and by exposure to social interaction. I therefore suggest that social neuroscience needs to make use of new experimental paradigms to systematically investigate this5. In conclusion, Rizzolatti and Sinigaglia provide an authoritative review of the involvement of the parieto-frontal cortex in action observation and action execution. However, they neglect recent conceptual and empirical developments that may have an important bearing on the interpretation of the relevant data. These developments suggest that future research should attempt to address how mechanisms that are perceived to exist in individuals might have to be re-assessed by taking social interaction seriously7–9. Such an attempt will help to shed light on the putatively complementary roles of mirror neurons and mentalizing networks as a function of direct engagement in realtime interaction and on aspects of cognition that might be unrelated to similarities between self-awareness and awareness of others in social interaction10.

What's in a smile? Neural correlates of facial embodiment during social interaction

Abstract Previous investigations have shown that the perception of socially relevant facial expressions, indicating someone elses intention to communicate (e.g., smiling), correlate with increased activity in zygomaticus major muscle regardless of whether the facial expressions seen are directed towards the human observer or toward someone else (Mojzisch et al., 2006). These spontaneous, involuntary reactions have been described as facial mimicry and seem to be of considerable importance for successful interpersonal communication. We investigated whether specific neural substrates underlie these responses by performing a finite impulse response (FIR) analysis of an experiment using functional magnetic resonance imaging (fMRI) to investigate the perception of socially relevant facial expressions (Schilbach et al., 2006). This analysis demonstrates that differential neural activity can be detected relative to the FIR time window in which facial mimicry occurs. The neural network found includes but extends beyond classical motor regions (face motor area) recruiting brain regions known to be involved in social cognition. This network is proposed to subserve the integration of emotional and action-related processes as part of a pre-reflective, embodied reaction to the perception of socially relevant facial expressions as well as a reflective representation of self and other.

From gaze cueing to dual eye-tracking: Novel approaches to investigate the neural correlates of gaze in social interaction

Tracking eye-movements provides easy access to cognitive processes involved in visual and sensorimotor processing. More recently, the underlying neural mechanisms have been examined by combining eye-tracking and functional neuroimaging methods. Apart from extracting visual information, gaze also serves important functions in social interactions. As a deictic cue, gaze can be used to direct the attention of another person to an object. Conversely, by following other persons gaze we gain access to their attentional focus, which is essential for understanding their mental states. Social gaze has therefore been studied extensively to understand the social brain. In this endeavor, gaze has mostly been investigated from an observational perspective using static displays of faces and eyes. However, there is growing consent that observational paradigms are insufficient for an understanding of the neural mechanisms of social gaze behavior, which typically involve active engagement in social interactions. Recent methodological advances have allowed increasing ecological validity by studying gaze in face-to-face encounters in real-time. Such improvements include interactions using virtual agents in gaze-contingent eye-tracking paradigms, live interactions via video feeds, and dual eye-tracking in two-person setups. These novel approaches can be used to analyze brain activity related to social gaze behavior. This review introduces these methodologies and discusses recent findings on the behavioral functions and neural mechanisms of gaze processing in social interaction.

The effects of self-involvement on attention, arousal, and facial expression during social interaction with virtual others: A psychophysiological study

Abstract Social neuroscience has shed light on the underpinnings of understanding other minds. The current study investigated the effect of self-involvement during social interaction on attention, arousal, and facial expression. Specifically, we sought to disentangle the effect of being personally addressed from the effect of decoding the meaning of another persons facial expression. To this end, eye movements, pupil size, and facial electromyographic (EMG) activity were recorded while participants observed virtual characters gazing at them or looking at someone else. In dynamic animations, the virtual characters then displayed either socially relevant facial expressions (similar to those used in everyday life situations to establish interpersonal contact) or arbitrary facial movements. The results show that attention allocation, as assessed by eye-tracking measurements, was specifically related to self-involvement regardless of the social meaning being conveyed. Arousal, as measured by pupil size, was primarily related to perceiving the virtual characters gender. In contrast, facial EMG activity was determined by the perception of socially relevant facial expressions irrespective of whom these were directed towards.