Carlotta Schatten

Electroencephalogram and Physiological Signal Analysis for Assessing Flow in Games

Passive brain-computer interaction (BCI) can provide useful information to understand a users state and anticipate intentions, which is needed to support adaptivity and personalization. Given the huge variety of audiences, a games capability of adapting to different user profiles-in particular to keep the player in flow-is crucial to make it ever more enjoyable and satisfying. We have performed a user experiment exploiting a four-electrode electroencephalogram (EEG) tool similar to the ones that are soon likely to appear in the market for game control. We have performed a spectral characterization of the video-gaming experience, also in comparison with other tasks. Results show that the most informative frequency bands for discriminating among gaming conditions are around low beta. Simple signals from the peripheral nervous system add marginal information. Classification of three levels of user states is possible, with good accuracy, using a support vector machine (SVM) classifier. A user-independent classification performs worse than a user-dependent approach (50.1% versus 66.4% rate). Personalized SVM training and validation time is reasonable (7-8 min). Thus, we argue that a personalized system could be implemented in a consumer context and research should aim at improving classifiers that can be trained online by end users.

Feature Analysis for Affect Recognition Supporting Task Sequencing in Adaptive Intelligent Tutoring Systems

Originally, the task sequencing in adaptive intelligent tutoring systems needs information gained from expert and domain knowledge as well as information about former performances. In a former work a new efficient task sequencer based on a performance prediction system was presented, which only needs former performance information but not the expensive expert and domain knowledge. This task sequencer uses the output of the performance prediction to sequence the tasks according to the theory of Vygotskys Zone of Proximal Development. In this paper we aim to support this sequencer by a further automatically to gain information source, namely speech input from the students interacting with the tutoring system. The proposed approach extracts features from students speech data and applies to that features an automatic affect recognition method. The output of the affect recognition method indicates, if the last task was too easy, too hard or appropriate for the student. Hence, as according to Vygotskys theory the next task should not be too easy or too hard for the student to neither bore nor frustrate him, obviously the output of our proposed affect recognition is suitable to be used as an input for supporting a sequencer based on the theory of Vygotskys Zone of Proximal Development. Hence, in this paper we (1) propose a new approach for supporting task sequencing by affect recognition, (2) present an analysis of appropriate features for affect recognition extracted from students speech input and (3) show the suitability of the proposed features for affect recognition for supporting task sequencing in adaptive intelligent tutoring systems.

Adaptive Content Sequencing without Domain Information

In this paper we show that a performance prediction method can be used to sequence contents ameliorating sequencing over common domain informed strategies. Our developed sequencer is able to sequence content without knowledge on the domain, i.e. without knowing the number of skills involved in a content and the relative difficulties. Moreover, we discuss if a synthetic learning process can be modeled in a plausible way in order to facilitate testing with sequencing, which is generally difficult to evaluate on real students. In conclusion we show that Matrix Factorization is able to deal with all the important actual problems of Intelligent Tutoring Systems: personalization, multiple skill content modeling, and difficulty.

Exploiting Real-Time EEG Analysis for Assessing Flow in Games

Ability of adapting to different users is considered key to maximize the impact and effectiveness of Serious Games (SGs). Advances in neurosciences are making it possible to continuously monitor the player status. This paper reports the research work on the monitoring of the player flow status with a simple (4 electrode) commercial electroencephalogram (EEG). In particular, we focus on three consequential research questions: is it possible to statistically distinguish a flow from a boredom condition? For which wavelengths? Can different levels of boredom and flow be identified? Results - even if limited because of the small size of the test, are promising and enable further research. Statistically significant differences could be observed for brainwaves under various conditions. A machine learning classifier (SVM) was successful in the 2-level distinction case, in particular with personalized training.

HNNP - A Hybrid Neural Network Plait for Improving Image Classification with Additional Side Information

Most of the artificial intelligence and machine learning researches deal with big data today. However, there are still a lot of real world problems for which only small and noisy data sets exist. Hence, in this paper we focus on those small data sets of noisy images. Applying learning models to such data may not lead to the best possible results because of few and noisy training examples. We propose a hybrid neural network plait for improving the classification performance of state-of-the-art learning models applied to the images of such data sets. The improvement is reached by (1) using additionally to the images different further side information delivering different feature sets and requiring different learning models, (2) retraining all different learning models interactively within one common structure. The proposed hybrid neural network plait architecture reached in the experiments with 2 different data sets on average a classification performance improvement of 40% and 52% compared to a single convolutional neural network and 13% and 17% compared to a stacking ensemble method.

Perceived Task-Difficulty Recognition from Log-file Information for the Use in Adaptive Intelligent Tutoring Systems

Recognising students’ emotion, affect or cognition is a relatively young field and still a challenging task in the area of intelligent tutoring systems. There are several ways to use the output of these recognition tasks within the system. The approach most often mentioned in the literature is using it for giving feedback to the students. The features used for that approach can be high-level features like linguistics features which are words related to emotions or affects, taken e.g. from written or spoken inputs, or low-level features like log-file features which are created from information contained in the log-files. In this work we aim at supporting task sequencing by perceived task-difficulty recognition on low-level features easily extracted from the log-file. We analyse these features by statistical tests showing that there are statistically significant feature combinations and hence the presented features are able to describe students’ perceived task-difficulty in intelligent tutoring systems. Furthermore, we apply different classification methods to the log-file features for perceived task-difficulty recognition and present a kind of higher ensemble method for improving the classification performance on the features extracted from a real data set. The presented approach outperforms classical ensemble methods and is able to improve the classification performance substantially, enabling a perceived task-difficulty recognition satisfactory enough for employing its output for components of a real system like task independent support or task sequencing.

An SVM Plait for Improving Affect Recognition in Intelligent Tutoring Systems

Usually, in intelligent tutoring systems the task sequencing is done by means of expert and domain knowledge. In a former work we presented a new efficient task sequencer without using the expensive expert and domain knowledge. This task sequencer only uses former performances and decides about the next task according to Vygotskys Zone of Proximal Development, that is to neither bore nor frustrate the student. We aim to support this task sequencer by a further automatically to gain information, namely students affect recognized from his speech input. However, the collection of the data from children needed for training an affect recognizer in this field is challenging as it is costly and complex and one has to consider privacy issues carefully. These problems lead to small data sets and limited performances of classification methods. Hence, in this work we propose an approach for improving the affect recognition in intelligent tutoring systems, which uses a special structure of several support vector machines with different input feature vectors. Furthermore, we propose a new kind of features for this problem. Different experiments with two real data sets show, that our approach is able to improve the classification performance on average by 49% in comparison to using a single classifier.

Automatic Subclasses Estimation for a Better Classification with HNNP

Although nowadays many artificial intelligence and especially machine learning research concerns big data, there are still a lot of real world problems for which only small and noisy data sets exist. Applying learning models to those data may not lead to desirable results. Hence, in a former work we proposed a hybrid neural network plait (HNNP) for improving the classification performance on those data. To address the high intraclass variance in the investigated data we used manually estimated subclasses for the HNNP approach. In this paper we investigate on the one hand the impact of using those subclasses instead of the main classes for HNNP and on the other hand an approach for an automatic subclasses estimation for HNNP to overcome the expensive and time consuming manual labeling. The results of the experiments with two different real data sets show that using automatically estimated subclasses for HNNP delivers the best classification performance and outperforms also single state-of-the-art neural networks as well as ensemble methods.

Minimal Invasive Integration of Learning Analytics Services in Intelligent Tutoring Systems

A common problem when trying to apply data mining techniques to improve educational systems is the disconnection between those who have the expertise (e.g. Universities) and those who have access to the data (e.g. Small companies). Bringing expertise into educational in-production systems is complicated because companies are reluctant to invest a lot of effort into integrating new technology that they do not fully trust, while the technology cannot prove its worth without access to real, valid data. In this paper we explore the requirements that machine learning systems have to be applied to specific learning problems (sequencing and performance prediction), and then propose a minimally invasive protocol for sequencing (based on web services) to easily integrate Learning Analytics Services into e-learning systems.

Local Feature Extractors Accelerating HNNP for Phoneme Recognition

Artificial neural networks are fast in the application phase but very slow in the training phase. On the other hand there are state-of-the-art approaches using neural networks, which are very efficient in image classification tasks, like the hybrid neural network plait (HNNP) approach for images from signal data stemming for instance from phonemes. We propose to accelerate HNNP for phoneme recognition by substituting the neural network with the highest computation costs, the convolutional neural network, within the HNNP by a preceding local feature extractor and a simpler and faster neural network. Hence, in this paper we propose appropriate feature extractors for this problem and investigate and compare the resulting computation costs as well as the classification performance. The results of our experiments show that HNNP with the best one of our proposed feature extractors in combination with a smaller neural network is more than two times faster than HNNP with the more complex convolutional neural network and delivers still a good classification performance.