Simulation of Facial Palsy using Conditional Generative Adversarial Networks

Abstract

Facial palsy is a serious disease; patients of facial palsy lose control of their facial muscles owing to the palsy of the facial nerve. The Yanagihara method is popularly used to evaluate facial palsy in Japan. In this method, patients are asked to make 10 different facial expressions, and doctors score each expression. However, different doctors, particularly those with limited experience, score the facial expressions differently in this method. Examples of scored images are important for educational purposes and as references. However, it is not possible to use facial images of the patients with facial palsy owing to concerns regarding the medical privacy of the patients. To solve this problem, we propose a facial palsy simulation method through which virtual facial images that are similar to those of the expressions of patients with facial palsy can be generated. We extract the facial expression information from patients and reconstruct faces with different identities using the information. In addition, we use facial boundaries as facial expression information and introduce a boundary-to-face network based on adversarial training.