Lawrence Ian Reed

Automatic analysis and recognition of brow actions and head motion in spontaneous facial behavior

Previous efforts in automatic facial expression recognition have been limited to posed facial behavior under well-controlled conditions (e.g., frontal pose and minimal out-of-plane head motion). The CMU/Pitt automated facial image analysis system (AFA) accommodates varied pose, moderate out-of-plane head motion, and occlusion. AFA was tested in video of two-person interviews originally collected to answer substantive questions in psychology, and represent a substantial challenge to automatic recognition of facial expression. This report focuses on two action units, brow raising and brow lowering because of their importance to emotion expression and paralinguistic communication. For two-state recognition, AFA achieved 89% accuracy. For three-state recognition (brow raising, brow lowering, and no brow action), accuracy was 76%. Brow and head motion were temporally coordinated. These findings demonstrate the feasibility of action unit recognition in spontaneous facial behavior.

Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles

Both the configuration of facial features and the timing of facial actions are important to emotion and communication. Previous literature has focused on the former. We developed an automatic facial expression analysis system that quantifies the timing of facial actions as well as head and eye motion during spontaneous facial expression. To assess coherence among these modalities, we recorded and analyzed spontaneous smiles in 62 young women of varied ethnicity ranging in age from 18 to 35 years. Spontaneous smiles occurred following directed facial action tasks, a situation likely to elicit spontaneous smiles of embarrassment. Smiles (AU 12) were manually FACS coded by certified FACS coders. The 3D head motion was recovered using a cylindrical head model. The motion vectors for lip-corner displacement were measured using feature-point tracking. The eye closure and the horizontal and vertical eye motion (from which to infer direction of gaze or visual regard) were measured by the generative model fitting approach. The mean correlation within subjects between lip-corner displacement, head motion, and eye motion ranged from +/0.36 to 0.50, which suggests moderate coherence among these features. Lip-corner displacement and head pitch were negatively correlated, as predicted for smiles of embarrassment. These findings are consistent with recent research in psychology suggesting that facial actions are embedded within coordinated motor structures. They suggest that the direction of correlation among features may discriminate between facial actions with similar morphology but different communicative meaning, inform automatic facial expression recognition, and provide normative data for animating computer avatars.

Anger, preoccupied attachment, and domain disorganization in borderline personality disorder

Emotional dysregulation and attachment insecurity have been reported in borderline personality disorder (BPD). Domain disorganization, evidenced in poor regulation of emotions and behaviors in relation to the demands of different social domains, may be a distinguishing feature of BPD. Understanding the interplay between these factors may be critical for identifying interacting processes in BPD and potential subtypes of BPD. Therefore, we examined the joint and interactive effects of anger, preoccupied attachment, and domain disorganization on BPD traits in a clinical sample of 128 psychiatric patients. The results suggest that these factors contribute to BPD both independently and in interaction, even when controlling for other personality disorder traits and Axis I symptoms. In regression analyses, the interaction between anger and domain disorganization predicted BPD traits. In recursive partitioning analyses, two possible paths to BPD were identified: high anger combined with high domain disorganization and low anger combined with preoccupied attachment. These results may suggest possible subtypes of BPD or possible mechanisms by which BPD traits are established and maintained.

The Commitment Function of Angry Facial Expressions

What function do facial expressions have? We tested the hypothesis that some expressions serve as honest signals of subjective commitments—in particular, that angry faces increase the effectiveness of threats. In an ultimatum game, proposers decided how much money to offer a responder while seeing a film clip depicting an angry or a neutral facial expression, together with a written threat that was either inherently credible (a 50-50 split) or less credible (a demand for 70% of the money). Proposers offered greater amounts in response to the less credible threat when it was accompanied by an angry expression than when it was accompanied by a neutral expression, but were unaffected by the expression when dealing with the credible threat. This finding supports the hypothesis that angry expressions are honest signals that enhance the credibility of threats.

The course of dysphoric affective and cognitive states in borderline personality disorder: a 10-year follow-up study.

The current study aimed to assess dysphoric states among 290 patients with borderline personality disorder (BPD) and 72 non-borderline axis II comparison subjects other personality disorders, (OPD) over a 10-year course of prospective follow-up. Additionally, we assessed the severity of these states among borderline patients who had and had not recovered both symptomatically and psychosocially. The Dysphoric Affect Scale (DAS) - a 50-item self-report measure of affective and cognitive states thought to be common among borderline patients and specific to the disorder - was administered at five waves of prospective follow-up. Affective and cognitive DAS items were separately analyzed, yielding respective subscores. Borderline patients reported more severe dysphoric states compared to OPD subjects at baseline. However, the severity of affective and cognitive states declined significantly for both groups taken together over 10 years of follow-up. Within the BPD group, recovered subjects reported less severe dysphoric states compared to non-recovered subjects at baseline. Results also showed a significant decline in DAS scores over time, but at a greater rate for recovered subjects. In sum, while the severity of dysphoric states declines significantly over time, inner distress remains an area of vulnerability for borderline subjects. Additionally, the severity and pervasiveness of these states may affect recovery over time.

The Communicative Function of Sad Facial Expressions

What are the communicative functions of sad facial expressions? Research shows that people feel sadness in response to losses but it’s unclear whether sad expressions function to communicate losses to others and if so, what makes these signals credible. Here we use economic games to test the hypothesis that sad expressions lend credibility to claims of loss. Participants play the role of either a proposer or recipient in a game with a fictional backstory and real monetary payoffs. The proposers view a (fictional) video of the recipient’s character displaying either a neutral or sad expression paired with a claim of loss. The proposer then decided how much money to give to the recipient. In three experiments, we test alternative theories by using situations in which the recipient’s losses were uncertain (Experiment 1), the recipient’s losses were certain (Experiment 2), or the recipient claims failed gains rather than losses (Experiment 3). Overall, we find that participants gave more money to recipients who displayed sad expressions compared to neutral expressions, but only under conditions of uncertain loss. This finding supports the hypothesis that sad expressions function to increase the credibility of claims of loss.

The Relationship Between Childhood Adversity and Dysphoric Inner States Among Borderline Patients Followed Prospectively for 10 Years

Childhood experiences of abuse and neglect were assessed in relation to dysphoric states among patients with borderline personality disorder (BPD) over a 10-year course of prospective follow-up. The Revised Childhood Experiences Questionnaire was administered at baseline to 290 patients meeting DIB-R and DSM-III-R criteria for BPD. The Dysphoric Affect Scale--a 50-item self-report measure of affective and cognitive states thought to be common among and specific to borderline patients--was administered at fives waves of prospective follow-up. Significant predictors of dysphoric states included emotional abuse, verbal abuse, physical abuse, sexual abuse, emotional withdrawal, inconsistent treatment, denial of patients feelings, lack of a real relationship, placing patient in parental role, and failure to protect patient. This suggests that abusive and neglectful childhood experiences are significant risk factors for severe affective and cognitive difficulties reported by borderline patients and that sexual abuse is neither necessary nor sufficient for the development of these troubling inner states.

The Emotional Moves of a Rational Actor: Smiles, Scowls, and Other Credible Messages

Many scholars turn to emotions to understand irrational behavior. We do the opposite: we turn to rationality and game theory to understand people’s emotions. We discuss a striking theory of emotions that began with the game theory of credible threats and promises, then was enriched by evolutionary biology and psychology, and now is being tested in psychological experiments. We review some of these experiments which use economic games to set up strategic situations with real payoffs. The experiments test whether a player’s emotional expressions lend credibility to promises, threats, and claims of danger or hardship. The results offer insights into the hidden strategies behind a warm smile, an angry scowl, a look of terror, and eyes of despair.

Effects of Tearing on the Perception of Facial Expressions of Emotion

What is the function of emotional tearing? Previous work has found a tear effect, which resolves ambiguity in neutral expressions and increases perceptions of sadness in sad expressions. Tearing, however, is associated with a variety of emotional states, and it remains unclear how the tear effect generalizes to other emotion expressions. Here we expand upon previous works by examining ratings of video clips depicting posed facial expressions presented with and without tears. We replicate Provine et al.’s (2009) findings that tearing increases perceptions of sadness in sad expressions. Furthermore, we find that tearing has specific effects on ratings of emotion (happiness, sadness, anger, and fear) and ratings of intensity and valence in neutral, positive, and negative expressions. These results suggest that tearing may serve a specific and independent communicative function, interacting with those of various expressions.

Sexual orientation in males and the evolution of anisogamy.

How might homosexual orientation have evolved and been maintained? Several adaptationist explanations have been examined in attempt to reconcile the presence of same-sex sexual behaviors with traditional selection-based theory, showing little empirical support. The current paper presents a novel adaptationist explanation for the evolution and maintenance of same-sex sexual behaviors in males, both between- and within-species, related to the evolution of anisogamy. Under conditions of isogamy, sexual reproduction occurs between individuals with gametes of similar morphology. With the evolution of anisogamy came greater specificity on the types of individuals that would produce offspring when mated with (i.e. those with opposing gamete sizes). It is suggested that with this evolutionary change, a specified psychological adaptation orienting individuals primarily towards mating partners with newly opposing gamete sizes was then selected for. It is thus hypothesized that sexual orientation will vary along the anisogamy-isogamy continuum, with homosexual orientation being associated with closer approximations towards isogamy. This hypothesis leads to two specific predictions. First, in comparisons between species, the presence of same-sex sexual behaviors will be more likely to occur as sperm to egg ratios approach 1:1. Second, in comparisons within species, those individuals with greater sperm lengths will be more likely to exhibit same-sex sexual behaviors than those with lesser sperm lengths. Examination of the present hypothesis stands to greatly increase our knowledge of the selective forces shaping both biological and psychological evolution.