Roman Gorbunov

Automatic Interpretation of Affective Facial Expressions in the Context of Interpersonal Interaction

This paper proposes a method for interpretation of the emotions detected in facial expressions in the context of the events that cause them. The method was developed to analyze the video recordings of facial expressions depicted during a collaborative game played as a part of the Mars-500 experiment. In this experiment, six astronauts were isolated for 520 days in a space station to simulate a flight to Mars. Seven time-dependent components of facial expressions were extracted from the video recordings of the experiment. To interpret these dynamic components, we proposed a mathematical model of emotional events. Genetic programming was used to find the locations, types, and intensities of the emotional events as well as the way the recorded facial expressions represented reactions to them. By classification of different statistical properties of the data, we found that there are significant relations between the facial expressions of different crew members and a memory effect between the collective emotional states of the crew members. The model of emotional events was validated on previously unseen video recordings of the astronauts. We demonstrated that both genetic search and optimization of the parameters improve the accuracy of the proposed model. This method is a step toward automating the analysis of affective expressions in terms of the cognitive appraisal theory of emotion, which relies on the dependence of the expressed emotion on the causing event.

Nonverbal behavior observation: collaborative gaming method for prediction of conflicts during long-term missions

This paper presents a method for monitoring mental state of small isolated crews during long-term missions (such as space mission, polar expeditions, submarine crews, meteorological stations, and etc). It combines the records of negotiation game with monitoring of the nonverbal behavior of the players. We analyze the records of negotiation game that has taken place between the crew members who were placed in isolated environment for 105 days during the Mars-500 experiment. The outcomes of the analysis, differently from the previously made conclusions, show that there was not a significant deviation of the rational choice of the players. We propose an extension of the method that includes monitoring of the nonverbal behavior of the players next to recording the game records. The method is focused on those aspects of psychological and sociological states that are crucial for the performance of the crew. In particular, we focus on measuring of emotional stress, initial signs of conflicts, trust, and ability to collaborate.

Automatic mental heath assistant: monitoring and measuring nonverbal behavior of the crew during long-term missions

This paper presents a method for monitoring the mental state of small isolated crews during long-term missions (such as space mission, polar expeditions, submarine crews, meteorological stations, and etc.) The research is done as a part of Automatic Mental Health Assistant (AMHA) project which aims to develop set of techniques for automatic measuring of intra- and inter- personal states in working groups. The method is focused on those aspects of psychological and sociological states that are crucial for the performance of the crew. In particular, we focus on measuring of emotional stress, initial signs of conflicts, trust, and ability to collaborate. The present research is performed in collaboration with MARS-500 experiment in which a small group of people is isolated for a long period of time. The MARS-500 experiment, in this way, provides a unique platform for study of human-human interaction. The confinement study will imitate all key peculiarities expected to be present during future missions to Mars (i.e. ultra long duration flight, need for autonomy, complicated communication with a digital communication center due to signal delay, and limited stock of expendables). The developed method is also currently tested by usage of a web-based platform.

Design of social agents

Social behavior, as compared to the egoistic and rational behavior, is known to be more beneficial to groups of subjects and even to individual members of a group. For this reason, social norms naturally emerge as a product of evolution in human and animal populations. The benefit of the social behavior makes it also an interesting subject in the field of artificial agents. Social interactions implemented in computer agents can improve their personal and group performance. In this study we formulate design principles of social agents and use them to create social computer agents. To construct social agents we take two approaches. First, we construct social computer agents based on our understanding of social norms. Second, we use an evolutionary approach to create social agents. The social agents are shown to outperform agents that do not utilize social behavior.

Memory effect in expressed emotions during long term group interactions

Long-term interactions in groups can be monitored through games in which the participants need to show their social preferences by making choice to help or to use egoistic game strategy. In this paper we analyse the facial expressions of a group of isolated individuals (astronauts) during repeated interactions in subsequent encounters in a game. The astronauts were taking part in the Mars-500 isolation experiment and their relations were influenced by the everyday interaction in this untypical environment, and monitored through the cooperative game. We analysed different statistical properties of the recorded emotional facial expressions of the astronauts, where emotions were determined by the FaceReader software. We found that there is a memory effect between the collective emotional expressions corresponding to subsequent experiments, separated by two weeks time period. This dependance suggest that it is possible to predict the development of interpersonal relations in groups of isolated individuals. In a broader perspective, this finding can inform the design of long-term interaction behavior of artificial agents.