Magdalena Rychlowska

Fashioning the Face: Sensorimotor Simulation Contributes to Facial Expression Recognition

When we observe a facial expression of emotion, we often mimic it. This automatic mimicry reflects underlying sensorimotor simulation that supports accurate emotion recognition. Why this is so is becoming more obvious: emotions are patterns of expressive, behavioral, physiological, and subjective feeling responses. Activation of one component can therefore automatically activate other components. When people simulate a perceived facial expression, they partially activate the corresponding emotional state in themselves, which provides a basis for inferring the underlying emotion of the expresser. We integrate recent evidence in favor of a role for sensorimotor simulation in emotion recognition. We then connect this account to a domain-general understanding of how sensory information from multiple modalities is integrated to generate perceptual predictions in the brain.

Heterogeneity of long-history migration explains cultural differences in reports of emotional expressivity and the functions of smiles

Significance In an age of globalization, emotional understanding is the central problem of human interaction. Here, we show that historical heterogeneity, or the extent to which a country’s present-day population descends from numerous (vs. few) source countries, predicts cultural variation in norms for emotional expressivity. Reanalysis of cultural display rules from 32 countries reveals that historical heterogeneity is associated with norms favoring greater emotional expressivity. In addition, the results of a study of nine countries show that the belief that smiles signal social bonding motives vs. the negotiation of status in a social hierarchy is predicted by historical heterogeneity as well. A small number of facial expressions may be universal in that they are produced by the same basic affective states and recognized as such throughout the world. However, other aspects of emotionally expressive behavior vary widely across culture. Just why do they vary? We propose that some cultural differences in expressive behavior are determined by historical heterogeneity, or the extent to which a country’s present-day population descended from migration from numerous vs. few source countries over a period of 500 y. Our reanalysis of data on cultural rules for displaying emotion from 32 countries [n = 5,340; Matsumoto D, Yoo S, Fontaine J (2008) J Cross Cult Psychol 39(1):55–74] reveals that historical heterogeneity explains substantial, unique variance in the degree to which individuals believe that emotions should be openly expressed. We also report an original study of the underlying states that people believe are signified by a smile. Cluster analysis applied to data from nine countries (n = 726), including Canada, France, Germany, India, Indonesia, Israel, Japan, New Zealand, and the United States, reveals that countries group into “cultures of smiling” determined by historical heterogeneity. Factor analysis shows that smiles sort into three social-functional subtypes: pleasure, affiliative, and dominance. The relative importance of these smile subtypes varies as a function of historical heterogeneity. These findings thus highlight the power of social-historical factors to explain cross-cultural variation in emotional expression and smile behavior.

Blocking Mimicry Makes True and False Smiles Look the Same

Recent research suggests that facial mimicry underlies accurate interpretation of subtle facial expressions. In three experiments, we manipulated mimicry and tested its role in judgments of the genuineness of true and false smiles. Experiment 1 used facial EMG to show that a new mouthguard technique for blocking mimicry modifies both the amount and the time course of facial reactions. In Experiments 2 and 3, participants rated true and false smiles either while wearing mouthguards or when allowed to freely mimic the smiles with or without additional distraction, namely holding a squeeze ball or wearing a finger-cuff heart rate monitor. Results showed that blocking mimicry compromised the decoding of true and false smiles such that they were judged as equally genuine. Together the experiments highlight the role of facial mimicry in judging subtle meanings of facial expressions.

Functional Smiles: Tools for Love, Sympathy, and War.

A smile is the most frequent facial expression, but not all smiles are equal. A social-functional account holds that smiles of reward, affiliation, and dominance serve basic social functions, including rewarding behavior, bonding socially, and negotiating hierarchy. Here, we characterize the facial-expression patterns associated with these three types of smiles. Specifically, we modeled the facial expressions using a data-driven approach and showed that reward smiles are symmetrical and accompanied by eyebrow raising, affiliative smiles involve lip pressing, and dominance smiles are asymmetrical and contain nose wrinkling and upper-lip raising. A Bayesian-classifier analysis and a detection task revealed that the three smile types are highly distinct. Finally, social judgments made by a separate participant group showed that the different smile types convey different social messages. Our results provide the first detailed description of the physical form and social messages conveyed by these three types of functional smiles and document the versatility of these facial expressions.

Body and Mind: Zajonc's (Re)introduction of the Motor System to Emotion and Cognition

Zajonc and Markus published a chapter in 1984 that proposed solutions to the difficult problem of modeling interactions between cognition and emotion. The most radical of their proposals was the importance of the motor system in information processing. These initial preoccupations, when wedded with the vascular theory of emotional efference (VTEE), propelled theory and research about how the face works to control emotion and to control interpersonal interaction. We discuss the development of Bob’s thinking about facial expression—facial efference is the term he preferred—as he moved toward predictions that sounded radical at the time, and which these days sound like precursors to advances in neuroscience and psychology subsumed under the term “embodied cognition.”

Negative Relations Between Pacifier Use and Emotional Competence

Research in psychology and neuroscience suggests that facial mimicry plays a causal role in understanding facial expression of emotion. Accurate understanding of facial emotion, in turn, grounds emotional development. Are pacifiers, which disrupt facial mimicry in the user, associated with compromised emotional development? We examined facial mimicry in children and found that duration of pacifier use was associated with reduced facial mimicry in boys. In two questionnaire studies of young adults, pacifier use also predicted lower perspective taking and emotional intelligence in males. Pacifier use did not predict these emotion processing skills in girls. Future confirmatory studies are proposed.

Feelings and contexts: socioecological influences on the nonverbal expression of emotion

Despite their relative universality, nonverbal displays of emotion are often sources of cross-cultural misunderstandings. The present article considers the relevance of historical and present socio-ecological contexts, such as heterogeneity of long-history migration, pathogen prevalence, and residential mobility for cross-cultural variation in emotional expression. We review recent evidence linking these constructs to psychological processes and discuss how the findings are relevant to the nonverbal communication of emotion. We hold that socioecological variables, because of their specificity and tractability, provide a promising framework for explaining why different cultures developed varying modes of emotional expression.

Towards 4D Coupled Models of Conversational Facial Expression Interactions.

In this paper we introduce a novel approach for building 4D coupled statistical models of conversational facial expression interactions. To build these coupled models we use 3D AAMs for feature extraction, 4D polynomial fitting for sequence representation, and concatenated feature vectors of frontchannel-backchannel interactions. Using a coupled model of conversation smile interactions, we predicted each sequence’s backchannel signal. In a subsequent experiment, human observers rated predicted backchannel sequences as highly similar to the originals. Our results demonstrate the usefulness of coupled models as powerful tools to analyse and synthesise key aspects of conversational interactions, including conversation timings, backchannel responses to frontchannel signals, and the spatial and temporal dynamics of conversational facial expression interactions. Methodology: Using a 4D database of natural, dyadic conversations [3], conversational interactions were manually annotated for conversational expressions. The sequences were tracked using a 4D sparse tracking approach, which uses 3D shape and texture.These tracked points are used as control points in a dense correspondence method. This method uses a Thin Plate Spline (TPS) based algorithm, with an additional “snapping” step, to modify the geometry of one mesh (reference mesh) so that it matches that of another mesh (target mesh). The tracking and intersubject registration methods were developed in-lab and details for these approaches can be found in [2]. Statistical modelling of these sequences was performed using 3D Active Appearance Models (AAMs) [1] and a polynomial regression technique for sequence representation. Experiments: In Experiment 1, individual sequences were classified as either frontchannel or backchannel. In Experiment 2, these sequences were also modified and used in a perceptual experiment that evaluated the realism of the synthesised sequences. For Experiments 3 and 4, a coupled statistical model of conversation interactions was built by concatenating the frontchannel sequence and corresponding backchannel sequence feature vectors (Table 1).

When the Good Guy Becomes the Bad Boy: Social Information Modulates the Neural, Physiological and Subjective Responses to Emotional Facial Expressions

In the past decade, different studies have suggested that high-order factors could influence the perceptual processing of emotional stimuli. In this study, we aimed to evaluate the effect of congruent vs. incongruent social information (positive, negative or no information related to the character of the target) on subjective (perceived and felt valence and arousal), physiological (facial mimicry) as well as on neural (P100 and N170) responses to dynamic emotoional facial expressions (EFE) that varied from neutral to one of the six basic emotions. Across three studies, the results showed (1) reduced valence and arousal evaluation of EFE when associated with incongruent social information (Study 1), (2) increased electromyographical responses (Study 2) and significant modulation of P100 and N170 components (Study 3) when EFE were associated with social (positive and negative) information (vs. no information). These studies revealed that positive or negative social information reduced subjective responses to incongruent EFE and produces a similar neural and physiological boost of the early perceptual processing of EFE irrespective of their congruency. In conclusion, this study suggested that social context (positive or negative) enhances the necessity to be alert to any subsequent cues.

The Perils of Boxes and Arrows: Reply to Gallese and Caruana

We recently [1] reviewed evidence indicating that sensorimotor simulation, and sometimes resulting facial mimicry, accompanies and supports the perception of facial expressions of emotion. In their comment (this issue), Gallese and Caruana [2] challenge the concept of what they call ‘expression/experience dualism’ and suggest that emotion experience, expression production, and expression recognition are inseparable processes that take place in a unified brain network that encompasses but is not restricted to sensorimotor systems.