Agata Kołakowska

Emotion recognition and its application in software engineering

In this paper a novel application of multimodal emotion recognition algorithms in software engineering is described. Several application scenarios are proposed concerning program usability testing and software process improvement. Also a set of emotional states relevant in that application area is identified. The multimodal emotion recognition method that integrates video and depth channels, physiological signals and input devices usage patterns is proposed and some preliminary results on learning set creation are described.

Emotion Recognition and Its Applications

This paper aims at illustrating diversity of possible emotion recognition applications. It provides concise review of affect recognition methods based on different inputs such as biometrics, video channel or behavioral data. It proposes a set of research scenarios of emotion recognition applications in the following domains: software engineering, website customization, education, and gaming. The scenarios show complexity and problems of applying affective computing in different domains. Analysis of the scenarios allows drawing some conclusions on challenges of automatic recognition that have to be addressed by further research.

Comparison of selected off-the-shelf solutions for emotion recognition based on facial expressions

The paper concerns accuracy of emotion recognition from facial expressions. As there are a couple of ready off-the-shelf solutions available in the market today, this study aims at practical evaluation of selected solutions in order to provide some insight into what potential buyers might expect. Two solutions were compared: FaceReader by Noldus and Xpress Engine by QuantumLab. The performed evaluation revealed that the recognition accuracies differ for photo and video input data and therefore solutions should be matched to the specificity of the application domain.

Recognizing emotions on the basis of keystroke dynamics

The article describes a research on recognizing emotional states on the basis of keystroke dynamics. An overview of various studies and applications of emotion recognition based on data coming from keyboard is presented. Then, the idea of an experiment is presented, i.e. the way of collecting and labeling training data, extracting features and finally training classifiers. Different classification approaches are proposed to be tested: universal vs. individual models, multiclass vs. two-class. The obtained results reveal which of these approaches are appropriate for the given task. The individual two-class models turn out to be the most accurate.

Applying decision trees to the recognition of musical symbols

The paper presents an experimental study on the recognition of printed musical scores. The first part of the study focuses on data preparation. Bitmaps containing musical symbols are converted to feature vectors using various methods. The vectors created in such a way are used to train classifiers which are the essential part of the study. Several decision tree classifiers are applied to this recognition task. These classifiers are created using different decision tree induction methods. The algorithms incorporate different criteria to select attributes in the nodes of the trees. Moreover, some of them apply stopping criteria, whereas the others perform tree pruning. The classification accuracy of the decision trees is estimated on data taken from musical scores. Eventually the usefulness of decision trees in the recognition of printed musical symbols is evaluated.

Automatic recognition of males and females among web browser users based on behavioural patterns of peripherals usage

Purpose The purpose of this paper is to answer the question whether it is possible to recognise the gender of a web browser user on the basis of keystroke dynamics and mouse movements. Design/methodology/approach An experiment was organised in order to track mouse and keyboard usage using a special web browser plug-in. After collecting the data, a number of parameters describing the users’ keystrokes, mouse movements and clicks were calculated for each data sample. Then several machine learning methods were used to verify the stated research question. Findings The experiment showed that it is possible to recognise males and females on the basis of behavioural characteristics with an accuracy exceeding 70 per cent. The best results were obtained while using Bayesian networks. Research limitations/implications The first limitation of the study was the restricted contextual information, i.e. neither the type of web page browsed nor the user activity was taken into account. Another is the narrow scope of the respondent group. Future work should focus on gathering data from more users covering a wider age range and should consider the context. Practical implications Automatic gender recognition could be used in profiling a user to create personalised websites or as an additional feature in automatic identification for security reasons. It might be also considered as a confirmation of declared gender in web-based surveys. Social implications As not all users perceive personalised ads and websites as beneficial, this application requires the analysis of a user perspective to provide value to the consumer without privacy violation. Originality/value Behavioural characteristics, such as mouse movements and keystroke dynamics, have already been used for user authentication and emotion recognition, but applying these data to gender recognition is an original idea.

Evaluation Criteria for Affect-Annotated Databases

In this paper a set of comprehensive evaluation criteria for affect-annotated databases is proposed. These criteria can be used for evaluation of the quality of a database on the stage of its creation as well as for evaluation and comparison of existing databases. The usefulness of these criteria is demonstrated on several databases selected from affect computing domain. The databases contain different kind of data: video or still images presenting facial expressions, speech recordings and affect-annotated words.

User Authentication Based on Keystroke Dynamics Analysis

In this paper a few methods used to authenticate users on the basis of their keystroke dynamics have been presented. Two of the methods define distance measures to compare typed texts. The third method examines feature distribution parameters. Some modification of one of the methods has been proposed. The methods have been tested and their performance compared on two data sets. Finally the three methods have been combined to generate another decision rule, which has been also compared with the three original ones.

Gyroscope-Based Game Revealing Progress of Children with Autism

The paper concerns the automation of measuring progress of children with autism spectrum disorder. The proposed approach combines diverse approaches: e-technologies and mobile applications for autism, behavioral metrics derived from gyroscope and game state with machine learning methods to find interconnections between the metrics and the progress of a child. The paper presents a gyroscope-based game, specifically designed as an investigation tool for therapy progress monitoring. The game enables registration of behavioral patterns of use of the applications and tablet. The paper presents how the game was used in a study of behavioral metrics. 31 children with autism took part in the study. Each of them played the game several times during a 6-months period. The data gathered during the gameplay are used to calculate a set of metrics, that might be used in evaluation of a childs progress. Results in terms of classification accuracy reach 80%, however they depend on the particular skill category. The best accuracies are obtained for evaluation of stereotypic behaviors and gross motor skills of a child. The approach presented in the study is novel and was not applied before, therefore it might be interesting for other researchers working on supporting technologies for autism. The results might be also interesting for practitioners applying e-technologies in autistics therapy.

Automatic recognition of therapy progress among children with autism

The article presents a research study on recognizing therapy progress among children with autism spectrum disorder. The progress is recognized on the basis of behavioural data gathered via five specially designed tablet games. Over 180 distinct parameters are calculated on the basis of raw data delivered via the game flow and tablet sensors - i.e. touch screen, accelerometer and gyroscope. The results obtained confirm the possibility of recognizing progress in particular areas of development. The recognition accuracy exceeds 80%. Moreover, the study identifies a subset of parameters which appear to be better predictors of therapy progress than others. The proposed method - consisting of data recording, parameter calculation formulas and prediction models - might be implemented in a tool to support both therapists and parents of autistic children. Such a tool might be used to monitor the course of the therapy, modify it and report its results.