Ian Sneddon

The HUMAINE Database: Addressing the Collection and Annotation of Naturalistic and Induced Emotional Data

The HUMAINE project is concerned with developing interfaces that will register and respond to emotion, particularly pervasive emotion (forms of feeling, expression and action that colour most of human life). The HUMAINE Database provides naturalistic clips which record that kind of material, in multiple modalities, and labelling techniques that are suited to describing it.

The Belfast Induced Natural Emotion Database

For many years psychological research on facial expression of emotion has relied heavily on a recognition paradigm based on posed static photographs. There is growing evidence that there may be fundamental differences between the expressions depicted in such stimuli and the emotional expressions present in everyday life. Affective computing, with its pragmatic emphasis on realism, needs examples of natural emotion. This paper describes a unique database containing recordings of mild to moderate emotionally colored responses to a series of laboratory-based emotion induction tasks. The recordings are accompanied by information on self-report of emotion and intensity, continuous trace-style ratings of valence and intensity, the sex of the participant, the sex of the experimenter, the active or passive nature of the induction task, and it gives researchers the opportunity to compare expressions from people from more than one culture.

Cross-Cultural Patterns in Dynamic Ratings of Positive and Negative Natural Emotional Behaviour

Background Studies of cross-cultural variations in the perception of emotion have typically compared rates of recognition of static posed stimulus photographs. That research has provided evidence for universality in the recognition of a range of emotions but also for some systematic cross-cultural variation in the interpretation of emotional expression. However, questions remain about how widely such findings can be generalised to real life emotional situations. The present study provides the first evidence that the previously reported interplay between universal and cultural influences extends to ratings of natural, dynamic emotional stimuli. Methodology/Principal Findings Participants from Northern Ireland, Serbia, Guatemala and Peru used a computer based tool to continuously rate the strength of positive and negative emotion being displayed in twelve short video sequences by people from the United Kingdom engaged in emotional conversations. Generalized additive mixed models were developed to assess the differences in perception of emotion between countries and sexes. Our results indicate that the temporal pattern of ratings is similar across cultures for a range of emotions and social contexts. However, there are systematic differences in intensity ratings between the countries, with participants from Northern Ireland making the most extreme ratings in the majority of the clips. Conclusions/Significance The results indicate that there is strong agreement across cultures in the valence and patterns of ratings of natural emotional situations but that participants from different cultures show systematic variation in the intensity with which they rate emotion. Results are discussed in terms of both ‘in-group advantage’ and ‘display rules’ approaches. This study indicates that examples of natural spontaneous emotional behaviour can be used to study cross-cultural variations in the perception of emotion.

The relationship between the stockperson's personality and attitudes and the productivity of dairy cows

This study investigated the relationships amongst personality traits and attitudes of 311 dairy stockpeople and the milk yield they obtained. A questionnaire pack consisting of a big-five measure of personality (which includes the traits of extraversion, agreeableness, conscientiousness, emotional stability and intellect), a four-factor attitude questionnaire and associated demographic and production questions was posted out to Northern Ireland dairy farmers. Pearson correlations were used to assess the relationship between personality and attitudes and partial correlations were calculated between milk yield and these psychometric measures. The personality traits of agreeableness and conscientiousness were most strongly correlated to positive attitudes towards working with dairy cows. None of the stockpeoples personality traits were significantly correlated with the milk yield they obtained. Three of the attitude scales, however, were significantly correlated with milk yield; milk yield was related to higher levels of empathy and job satisfaction and lower levels of negative beliefs. These findings, along with previous research, suggest stockperson attitudes may be important in relation to dairy cow welfare and production.

Gender Differences in the Perceptions of Genuine and Simulated Laughter and Amused Facial Expressions

This article addresses gender differences in laughter and smiling from an evolutionary perspective. Laughter and smiling can be responses to successful display behavior or signals of affiliation amongst conversational partners—differing social and evolutionary agendas mean there are different motivations when interpreting these signals. Two experiments assess perceptions of genuine and simulated male and female laughter and amusement social signals. Results show male simulation can always be distinguished. Female simulation is more complicated as males seem to distinguish cues of simulation yet judge simulated signals to be genuine. Females judge other female’s genuine signals to have higher levels of simulation. Results highlight the importance of laughter and smiling in human interactions, use of dynamic stimuli, and using multiple methodologies to assess perception.

The HUMAINE Database

The HUMAINE Database is grounded in HUMAINE’s core emphasis on considering emotion in a broad sense – ‘pervasive emotion’ – and engaging with the way it colours action and interaction. The aim of the database is to provide a resource to which the community can go to see and hear the forms that emotion takes in everyday action and interaction, and to look at the tools that might be relevant to describing it. Earlier chapters in this handbook describe the techniques and models underpinning the collection and labelling of such data. This chapter focuses on conveying the range of forms that emotion takes in the database, the ways that they can be labelled and the issues that the data raises. The HUMAINE Database provides naturalistic clips which record that kind of material, in multiple modalities, and labelling techniques that are suited to describing it. It was clear when the HUMAINE project began that work on databases should form part of it. However there were very different directions that the work might have taken. They were encapsulated early on in the contrast between ‘supportive’ and ‘provocative’ approaches, introduced in an earlier chapter in this handbook. The supportive option was to assemble a body of data whose size and structure allowed it to be used directly to build systems for recognition and/or synthesis. The provocative option was to assemble a body of data that encapsulated the challenges that the field faces.

Issues in Data Collection

The chapter reviews methods of obtaining records that show signs of emotion. Concern with authenticity is central to the task. Converging lines of argument indicate that even sophisticated acting does not reproduce emotion as it appears in everyday action and interaction. Acting is the appropriate source for some kinds of material, and work on that topic is described. Methods that aim for complete naturalism are also described, and the problems associated with them are noted. Techniques for inducing emotion are considered under five headings: classical induction; physical induction; games; task settings; and conversational interactions. The ethical issues that affect area are outlined, and a framework for dealing with them is set out.

Principles and History

Developing databases for emotion-oriented computing raises specific and complex issues at multiple levels, from the practicalities of recording to conceptual issues in psychology. Whether it is developing databases or using them, research in emotion-oriented computing needs to think about these issues rather than reflexively importing habits derived from other fields. Contemporary research identifies a number of principles that are relevant to making appropriate choices. They can be grouped under three broad headings – function; structure and scope; and relationship to psychological theory. These principles were not obvious when research in the area began. They have emerged gradually over a decade of relatively sustained work, and it is reviewed. Databases that have played a significant role in the process are listed, and selected case studies are examined in more depth. Lessons are drawn for future work at two levels, first at the level of an abstract overview and then at the level of practical issues that need to be addressed.