Karen L. Schmidt

Signal characteristics of spontaneous facial expressions: automatic movement in solitary and social smiles

The assumption that the smile is an evolved facial display suggests that there may be universal features of smiling in addition to the basic facial configuration. We show that smiles include not only a stable configuration of features, but also temporally consistent movement patterns. In spontaneous smiles from two social contexts, duration of lip corner movement during the onset phase was independent of social context and the presence of other facial movements, including dampening. These additional movements produced variation in both peak and offset duration. Both onsets and offsets had dynamic properties similar to automatically controlled movements, with a consistent relation between maximum velocity and amplitude of lip corner movement in smiles from two distinct contexts. Despite the effects of individual and social factors on facial expression timing overall, consistency in onset and offset phases suggests that portions of the smile display are relatively stereotyped and may be automatically produced.

Individual differences in facial expression: stability over time, relation to self-reported emotion, and ability to inform person identification

The face can communicate varied personal information including subjective emotion, communicative intent, and cognitive appraisal. Accurate interpretation by observer or computer interface depends on attention to dynamic properties of the expression, context, and knowledge of what is normative for a given individual. In two separate studies, we investigated individual differences in the base rate of positive facial expression and in specific facial action units over intervals from 4 to 12 months. Facial expression was measured using convergent measures, including facial EMG, automatic feature-point tracking, and manual FACS coding. Individual differences in facial expression were stable over time, comparable in magnitude to stability of self-reported emotion, and sufficiently strong that individuals were recognized on the basis of their facial behavior alone at rates comparable to that for a commercial face recognition system (Facelt from Identix). Facial action units convey unique information about person identity that can inform interpretation of psychological states, person recognition, and design of individuated avatars.

Dynamics of facial expression: normative characteristics and individual differences

Although the importance of facial expression in human computer interaction and in normal human interaction is widely acknowledged, there is very little data on the normative characteristics and stable individual differences for even the most common facial expressions. Dynamic characteristics of 195 spontaneous smiles from 95 individuals were measured using the facial action coding system, automated facial analysis and facial electromyography. Normative patterns observed included the characteristic timing of other facial actions with respect to action unit 12 (“smile”) and a mean duration of 15.7 frames for smile onset. Stable inter-individual differences included patterns of nonverbal actions associated with individuals’ smiles, and the amount of activity in the zygomaticus major muscle in two sessions recorded a year apart. These data are important in quantifying and fully describing individual differences in naturalistic human facial expression, as well as adding to our knowledge of spontaneous human smiles.

Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles

Both the configuration of facial features and the timing of facial actions are important to emotion and communication. Previous literature has focused on the former. We developed an automatic facial expression analysis system that quantifies the timing of facial actions as well as head and eye motion during spontaneous facial expression. To assess coherence among these modalities, we recorded and analyzed spontaneous smiles in 62 young women of varied ethnicity ranging in age from 18 to 35 years. Spontaneous smiles occurred following directed facial action tasks, a situation likely to elicit spontaneous smiles of embarrassment. Smiles (AU 12) were manually FACS coded by certified FACS coders. The 3D head motion was recovered using a cylindrical head model. The motion vectors for lip-corner displacement were measured using feature-point tracking. The eye closure and the horizontal and vertical eye motion (from which to infer direction of gaze or visual regard) were measured by the generative model fitting approach. The mean correlation within subjects between lip-corner displacement, head motion, and eye motion ranged from +/0.36 to 0.50, which suggests moderate coherence among these features. Lip-corner displacement and head pitch were negatively correlated, as predicted for smiles of embarrassment. These findings are consistent with recent research in psychology suggesting that facial actions are embedded within coordinated motor structures. They suggest that the direction of correlation among features may discriminate between facial actions with similar morphology but different communicative meaning, inform automatic facial expression recognition, and provide normative data for animating computer avatars.

Facial asymmetry quantification for expression invariant human identification

We investigate the effect of quantified statistical facial asymmetry as a biometric under expression variations. Our findings show that the facial asymmetry measures (AsymFaces) are computationally feasible, containing discriminative information and providing synergy when combined with Fisherface and Eigen-face methods on image data of two publically available face databases (Cohn-Kanade (T. Kanade et al., 1999) and Feret (P.J. Phillips et al., 1998)).

Peer-Mentored Research Development Meeting: A Model for Successful Peer Mentoring among Junior Level Researchers.

ObjectiveThis report describes a model for the development, process, and tracking methods of a Peer-mentored Research Development Meeting (PRDM), an interdisciplinary peer mentoring program. The program was initiated in 2004 by a group of post-doctoral scholars and junior faculty from the Schools of the Health Sciences at the University of Pittsburgh.MethodFrom February 2004 through February 2006, PRDM’s first five members tracked and documented their research activity (e.g., manuscripts, grants) every 4 months. The defining features of PRDM are adherence to a structured frequency and format for meetings, systematic tracking and evaluation of research development activities, and maintenance of ongoing relationships with senior mentors.ResultsDuring the 24-month data collection period, members were involved in 91 research development projects including grant applications, journal article manuscripts, book chapters, and conference abstracts. Members’ productivity increased during the 24-month period, as did the efficiency and focus of the completed projects.ConclusionMembers increased the efficiency and focus of their research development activities during the study period. Structured peer-mentoring groups have the potential to enhance research productivity among junior investigators in research intensive environments.

Automated facial image analysis: detecting improvement in abnormal facial movement after treatment with botulinum toxin A.

The purpose of this study was to evaluate the ability of Automated Facial Image Analysis (AFA) to detect changes in facial motion after Botox injections in patients with facial nerve disorders accompanied by abnormal muscle activity. Eight subjects received Botox for oral to ocular synkinesis (n = 6), ocular to oral synkinesis (n = 1), and/or depressor anguli oris overactivity (n = 3). Subjects were video-recorded during 2 directed facial action tasks before and after Botox treatment. AFA measurement and Facial Grading System (FGS) scores were used to evaluate the effects of Botox. After Botox, AFA detected a decrease in abnormal movements of the eyelids in all patients with oral to ocular synkinesis, a decrease in oral commissure movement for the patients with ocular to oral synkinesis, and an increase in oral commissure movement in all patients with depressor overactivity. The FGS scores failed to demonstrate any change in facial movement for the case of ocular to oral synkinesis and for 2 cases of depressor overactivity. AFA enables recognition of subtle changes in facial movement that may not be adequately measured by observer based ratings of facial function.

Verbal communication among Alzheimer's disease patients, their caregivers, and primary care physicians during primary care office visits

OBJECTIVE Primary care visits of patients with Alzheimers disease (AD) often involve communication among patients, family caregivers, and primary care physicians (PCPs). The objective of this study was to understand the nature of each individuals verbal participation in these triadic interactions. METHODS To define the verbal communication dynamics of AD care triads, we compared verbal participation (percent of total visit speech) by each participant in patient/caregiver/PCP triads. Twenty-three triads were audio taped during a routine primary care visit. Rates of verbal participation were described and effects of patient cognitive status (MMSE score, verbal fluency) on verbal participation were assessed. RESULTS PCP verbal participation was highest at 53% of total visit speech, followed by caregivers (31%) and patients (16%). Patient cognitive measures were related to patient and caregiver verbal participation, but not to PCP participation. Caregiver satisfaction with interpersonal treatment by PCP was positively related to caregivers own verbal participation. CONCLUSION Caregivers of AD patients and PCPs maintain active, coordinated verbal participation in primary care visits while patients participate less. PRACTICE IMPLICATIONS Encouraging verbal participation by AD patients and their caregivers may increase the AD patients active role and caregiver satisfaction with primary care visits.

Antisaccadic eye movements and attentional asymmetry in schizophrenia in three Pacific populations.

Antisaccadic eye movements were examined in 50 patients with schizophrenia and in 77 controls in three Pacific populations, namely New Zealand, Palau and Papua New Guinea. Despite the great biocultural variation encompassed by these three populations, schizophrenic patients made significantly more antisaccadic errors than controls (36% vs. 13%), as has been demonstrated previously in other populations. This neurocognitive deficit may be consistent with frontal lobe dysfunction in schizophrenia. In addition, patients with schizophrenia made significantly more errors than expected when the target was presented in the right visual field (RVF). This trend was observed in patients from all three study areas, and was not seen in any of the control populations. Antisaccadic test performance in schizophrenia may be influenced by lateralized (left hemisphere) neuroanatomical impairment.

Muscle activity in the partially paralyzed face after placement of a fascial sling: a preliminary report.

Neuromuscular re-education (ie, physical therapy) is often the first treatment given to patients with a partial facial paralysis. The purpose of this paper was to examine whether by repositioning and supporting partially paretic muscles with a fascial sling, one could improve facial movement in patients for whom the benefits of physical therapy had plateaued. Six patients with a history of unilateral, partial facial paralysis were assessed using the Facial Grading System (FGS) and surface electromyography (EMG) recordings of facial muscle activity. Automated facial analysis (AFA) was used to measure the facial excursions of the most recent patient. The FGS composite scores indicated improvement following static sling placement in all patients. The FGS subscale scores for voluntary movement indicated that the excursion of facial movement increased in 4 of the 6 patients. Surface EMG data demonstrated increased muscle activity in the zygomaticus major muscle in all patients. AFA demonstrated that following sling placement, the excursion of the lip commissure nearly doubled. The sling procedure, traditionally considered an intervention for improving static symmetry of the face, may also be useful for enhancing movement in some patients with a partial facial paralysis. Additional data, such as measurements provided by AFA, are needed to correlate facial displacement with EMG muscle activity.