Nurul Lubis

Emotion and Its Triggers in Human Spoken Dialogue: Recognition and Analysis

Human communication is naturally colored by emotion, triggered by the other speakers involved in the interaction. Therefore, to build a natural spoken dialogue system, it is essential to consider emotional aspects, which should be done not only by identifying user emotion, but also by investigating the reason why the emotion occurred. The ability to do so is especially important in situated dialogue, where the current situation plays a role in the interaction. In this paper, we propose a method of automatic recognition of emotion using support vector machine (SVM) and present further analysis regarding emotion triggers. Experiments were performed on an emotionally colorful dialogue corpus. The result shows performance that surpasses random guessing accuracy.

Emotion recognition on Indonesian television talk shows

As interaction between human and computer continues to develop to the most natural form possible, it becomes more and more urgent to incorporate emotion in the equation. The field continues to develop, yet exploration of the subject in Indonesian is still very lacking. This paper presents the first study of emotion recognition in Indonesian, including the construction of the first emotionally colored speech corpus in the language, and the building of an emotion classifier through an optimized machine learning process. We construct our corpus using television talk show recordings in various topics of discussion, yielding colorful emotional utterances. In our machine learning experiment, we employ the support vector machine (SVM) algorithm with feature selection and parameter optimization to ensure the best resulting model possible. Evaluation of the experiment result shows recognition accuracy of 68.31% at best.

Construction and analysis of Indonesian Emotional Speech Corpus

In this paper we present Indonesian Emotional Speech Corpus (IDESC), the first ever corpus in Indonesian that contains various emotion contents. As interaction between human and computer makes its way to the most natural form possible, it becomes more and more urgent to incorporate emotion in the equation. However, in Indonesian, this aspect is yet to be explored. The acquisition of an emotion corpus serves as a foundation in further research regarding the subject. In constructing IDESC, we aim at natural and real emotion that is applicable to human-computer interaction. The corpus consists of three episodes of Indonesian talk show in different genres: politics, humanity, and entertainment. Each episode is carefully segmented and labeled based on its emotion content, resulting in 2179 segments worth 1 hour, 34 minutes, and 49.7 seconds of speech. The corpus is still in its early stage of development, yielding exciting possibilities of future works.

A study of social-affective communication: Automatic prediction of emotion triggers and responses in television talk shows

Advancements in spoken language technologies have allowed users to interact with computers in an increasingly natural manner. However, most conversational agents or dialogue systems are yet to consider emotional awareness in interaction. To consider emotion in these situations, social-affective knowledge in conversational agents is essential. In this paper, we present a study of the social-affective process in natural conversation from television talk shows. We analyze occurrences of emotion (emotional responses), and the events that elicit them (emotional triggers). We then utilize our analysis for prediction to model the ability of a dialogue system to decide an action and response in an affective interaction. This knowledge has great potential to incorporate emotion into human-computer interaction. Experiments in two languages, English and Indonesian, show that automatic prediction performance surpasses random guessing accuracy.

Eliciting Positive Emotional Impact in Dialogue Response Selection

Introduction of emotion into human-computer interaction (HCI) have allowed various system’s abilities that can benefit the user. Among many is emotion elicitation, which is highly potential in providing emotional support. To date, works on emotion elicitation have only focused on the intention of elicitation itself, e.g. through emotion targets or personalities. In this paper, we aim to extend the existing studies by utilizing examples of human appraisal in spoken dialogue to elicit a positive emotional impact in an interaction. We augment the widely used example-based approach with emotional constraints: (1) emotion similarity between user query and examples, and (2) potential emotional impact of the candidate responses. Text-based human subjective evaluation with crowdsourcing shows that the proposed dialogue system elicits an overall more positive emotional impact, and yields higher coherence as well as emotional connection.

Construction and analysis of social-affective interaction corpus in English and Indonesian

Social-affective aspects of interaction play a vital role in making human communication a rich and dynamic experience. Observation of complex emotional phenomena requires rich sets of labeled data of natural interaction. Although there has been an increase of interest in constructing corpora containing social interactions, there is still a lack of spontaneous and emotionally rich corpora. This paper presents a corpus of social-affective interactions in English and Indonesian, constructed from various television talk shows, containing natural conversations and real emotion occurrences. We carefully annotate the corpus in terms of emotion and discourse structure to allow for the aforementioned observation. The corpus is still in its early stage of development, yielding wide-ranging possibilities for future work.