Ela Naz Döğer

Image-based Analysis of Emotional Facial Expressions in Full Face Transplants

In this study, it is aimed to determine the degree of the development in emotional expression of full face transplant patients from photographs. Hence, a rehabilitation process can be planned according to the determination of degrees as a later work. As envisaged, in full face transplant cases, the determination of expressions can be confused or cannot be achieved as the healthy control group. In order to perform image-based analysis, a control group consist of 9 healthy males and 2 full-face transplant patients participated in the study. Appearance-based Gabor Wavelet Transform (GWT) and Local Binary Pattern (LBP) methods are adopted for recognizing neutral and 6 emotional expressions which consist of angry, scared, happy, hate, confused and sad. Feature extraction was carried out by using both methods and combination of these methods serially. In the performed expressions, the extracted features of the most distinct zones in the facial area where the eye and mouth region, have been used to classify the emotions. Also, the combination of these region features has been used to improve classifier performance. Control subjects and transplant patients’ ability to perform emotional expressions have been determined with K-nearest neighbor (KNN) classifier with region-specific and method-specific decision stages. The results have been compared with healthy group. It has been observed that transplant patients don’t reflect some emotional expressions. Also, there were confusions among expressions.

Assessment of Emotional Expressions after Full-Face Transplantation

We assessed clinical features as well as sensory and motor recoveries in 3 full-face transplantation patients. A frequency analysis was performed on facial surface electromyography data collected during 6 basic emotional expressions and 4 primary facial movements. Motor progress was assessed using the wavelet packet method by comparison against the mean results obtained from 10 healthy subjects. Analyses were conducted on 1 patient at approximately 1 year after face transplantation and at 2 years after transplantation in the remaining 2 patients. Motor recovery was observed following sensory recovery in all 3 patients; however, the 3 cases had different backgrounds and exhibited different degrees and rates of sensory and motor improvements after transplant. Wavelet packet energy was detected in all patients during emotional expressions and primary movements; however, there were fewer active channels during expressions in transplant patients compared to healthy individuals, and patterns of wavelet packet energy were different for each patient. Finally, high-frequency components were typically detected in patients during emotional expressions, but fewer channels demonstrated these high-frequency components in patients compared to healthy individuals. Our data suggest that the posttransplantation recovery of emotional facial expression requires neural plasticity.

Recovery of facial expressions using functional electrical stimulation after full-face transplantation

BackgroundWe assessed the recovery of 2 face transplantation patients with measures of complexity during neuromuscular rehabilitation. Cognitive rehabilitation methods and functional electrical stimulation were used to improve facial emotional expressions of full-face transplantation patients for 5 months. Rehabilitation and analyses were conducted at approximately 3 years after full facial transplantation in the patient group. We report complexity analysis of surface electromyography signals of these two patients in comparison to the results of 10 healthy individuals.MethodsFacial surface electromyography data were collected during 6 basic emotional expressions and 4 primary facial movements from 2 full-face transplantation patients and 10 healthy individuals to determine a strategy of functional electrical stimulation and understand the mechanisms of rehabilitation. A new personalized rehabilitation technique was developed using the wavelet packet method. Rehabilitation sessions were applied twice a month for 5 months. Subsequently, motor and functional progress was assessed by comparing the fuzzy entropy of surface electromyography data against the results obtained from patients before rehabilitation and the mean results obtained from 10 healthy subjects.ResultsAt the end of personalized rehabilitation, the patient group showed improvements in their facial symmetry and their ability to perform basic facial expressions and primary facial movements. Similarity in the pattern of fuzzy entropy for facial expressions between the patient group and healthy individuals increased. Synkinesis was detected during primary facial movements in the patient group, and one patient showed synkinesis during the happiness expression. Synkinesis in the lower face region of one of the patients was eliminated for the lid tightening movement.ConclusionsThe recovery of emotional expressions after personalized rehabilitation was satisfactory to the patients. The assessment with complexity analysis of sEMG data can be used for developing new neurorehabilitation techniques and detecting synkinesis after full-face transplantation.

Actual and artificial facial expressions effect on heart rate variability

In this study, Heart Rate Variability (HRV) changes, caused by the actual and artificial emotional expressions, were analyzed. In order to evaluate this, EKG recordings have been recorded from eight young and healthy volunteers. Happiness, disgust and fear expression were selected for record. The expressions of emotions have been recorded with surface 4 channel EKG during these emotions. Recording was performed in 2 states by all volunteers. In first stage, volunteers were procured artificial emotional expressions without stimulation. In second stage volunteers were procured actual emotional expressions with visual stimulation (by watched short movies about related expression). Heart Rate Variability (HRV) values calculated from recorded EKG data, then interpolation and 4 Hz re-sampling processed. Then, power spectral density have been analyzed using Welch Periodogram. Spectral changes and the sempatovagal balance values discussed and interpreted by virtue of expressions.

Entropy analysis of surface EMG for classification of face movements

Measuring complexity of dynamical systems is a mighty tool for electrophysiological signal processing. There are plenty of entropies for estimating complexity measure. Approximate entropy (ApEn), sample entropy (SampEn), fuzzy entropy (FuzzyEn), wavelet entropy (WE) and wavelet packet entropy (WPE) was used for surface EMG feature extraction for face movements classification. Linear discriminant analysis (LDA) selected for classification. Classification performance was determined by mean square error (MSE) for different window sizes. Fuzzy entropy is the most robust and succeeding method of them. Principal component analysis used to improve classification performance however just results of approximate entropy feature were refined. MSE of wavelet entropy and wavelet packet entropy are also decent methods for this classification problem.

Image based analysis of emotional expression development in facial transplant patient

In this study, in healthy groups and facial transplantation patient, Gabor Wavelet transform based on appearance was adopted for recognition neutral and 6 emotional facial expressions consisting of angry, scared, happy, hate, confused and sad. The aim is to examine the ratio of regional performing facial mimics of face transplantation patient acording to healthy groups during each expressions. The system consists two phases; in the first stage, facial images of the healthy group were used for the training and test input, and in the second stage, the images of facial transplantation patient were used for the test input of trained system. Feature vectors were obtained from face images which convolved with Gabor filter (5-scales, 8-orientations) banks to feature extraction from the location of the eyes and mouth which is the most prominent region, while performing expressions. This feature vectors were classified by K-nearest neighbor classifier. The results were evaluated for normal groups and transplant patients.