Mariusz Szwoch

Emotion recognition and its application in software engineering

In this paper a novel application of multimodal emotion recognition algorithms in software engineering is described. Several application scenarios are proposed concerning program usability testing and software process improvement. Also a set of emotional states relevant in that application area is identified. The multimodal emotion recognition method that integrates video and depth channels, physiological signals and input devices usage patterns is proposed and some preliminary results on learning set creation are described.

Guido: A Musical Score Recognition System

This paper presents an optical music recognition system Guido that can automatically recognize the main musical symbols of music scores that were scanned or taken by a digital camera. The application is based on object model of musical notation and uses linguistic approach for symbol interpretation and error correction. The system offers musical editor with a partially automatic error correction.

FEEDB: A multimodal database of facial expressions and emotions

In this paper a first version of a multimodal FEEDB database of facial expressions and emotions is presented. The database contains labeled RGB-D recordings of people expressing a specific set of expressions that have been recorded using Microsoft Kinect sensor. Such a database can be used for classifier training and testing in face recognition as well as in recognition of facial expressions and human emotions. Also initial experiences on gathering evoked expressions and depth data processing are presented.

Emotion Recognition and Its Applications

This paper aims at illustrating diversity of possible emotion recognition applications. It provides concise review of affect recognition methods based on different inputs such as biometrics, video channel or behavioral data. It proposes a set of research scenarios of emotion recognition applications in the following domains: software engineering, website customization, education, and gaming. The scenarios show complexity and problems of applying affective computing in different domains. Analysis of the scenarios allows drawing some conclusions on challenges of automatic recognition that have to be addressed by further research.

On Facial Expressions and Emotions RGB-D Database

The goal of this paper is to present the idea of creating reference database of RGB-D video recordings for recognition of facial expressions and emotions. Two different formats of the recordings used for creation of two versions of the database are described and compared using different criteria. Examples of first applications using databases are also presented to evaluate their usefulness.

Comparison of selected off-the-shelf solutions for emotion recognition based on facial expressions

The paper concerns accuracy of emotion recognition from facial expressions. As there are a couple of ready off-the-shelf solutions available in the market today, this study aims at practical evaluation of selected solutions in order to provide some insight into what potential buyers might expect. Two solutions were compared: FaceReader by Noldus and Xpress Engine by QuantumLab. The performed evaluation revealed that the recognition accuracies differ for photo and video input data and therefore solutions should be matched to the specificity of the application domain.

Eye Blink Based Detection of Liveness in Biometric Authentication Systems Using Conditional Random Fields

The goal of this paper was to verify whether the conditional random fields are suitable and enough efficient for eye blink detection in user authentication systems based on face recognition with a standard web camera. To evaluate this approach several experiments were carried on using a specially developed test application and video database.

Using MusicXML to Evaluate Accuracy of OMR Systems

In this paper a methodology for automatic accuracy evaluation in optical music recognition (OMR) applications is proposed. Presented approach assumes using ground truth images together with digital music scores describing their content. The automatic evaluation algorithm measures differences between the tested score and the reference one, both stored in MusicXML format. Some preliminary test results of this approach are presented based on the algorithms implementation in OMR Guido application.

Emotion Recognition for Affect Aware Video Games

In this paper the idea of affect aware video games is presented. A brief review of automatic multimodal affect recognition of facial expressions and emotions is given. The first result of emotions recognition using depth data as well as prototype affect aware video game are presented.

Facial emotion recognition using depth data

In this paper an original approach is presented for facial expression and emotion recognition based only on depth channel from Microsoft Kinect sensor. The emotional user model contains nine emotions including the neutral one. The proposed recognition algorithm uses local movements detection within the face area in order to recognize actual facial expression. This approach has been validated on Facial Expressions and Emotions Database using 169 recordings of 25 persons. Though an average recognition accuracy is slightly above 50% this approach is highly independent of illumination conditions and also accepts low distance between sensor and the user. Thus, the proposed approach can be used to support other algorithms based on optical channel, as well as using skeleton or face tracking information.