Olga Sourina

Real-Time EEG-Based Human Emotion Recognition and Visualization

Emotions accompany everyone in the daily life, playing a key role in non-verbal communication, and they are essential to the understanding of human behavior. Emotion recognition could be done from the text, speech, facial expression or gesture. In this paper, we concentrate on recognition of “inner” emotions from electroencephalogram (EEG) signals as humans could control their facial expressions or vocal intonation. The need and importance of the automatic emotion recognition from EEG signals has grown with increasing role of brain computer interface applications and development of new forms of human-centric and human-driven interaction with digital media. We propose fractal dimension based algorithm of quantification of basic emotions and describe its implementation as a feedback in 3D virtual environments. The user emotions are recognized and visualized in real time on his/her avatar adding one more so-called “emotion dimension” to human computer interfaces.

EEG-Based "Serious" Games Design for Medical Applications

Recently, EEG-based technology has become more popular in “serious” games designs and developments since new wireless headsets that meet consumer demand for wear ability, price, portability and ease-of-use are coming to the market. Originally, EEG-based technologies were used in neurofeedback games and brain-computer interfaces. Now, such technologies could be used in entertainment, e-learning and new medical applications. In this paper, we review on neurofeedback game designs and algorithms, and propose design, algorithm, and implementation of new EEG-based 2D and 3D concentration games. Possible future medical applications of the games are discussed.

Fractal dimension based neurofeedback in serious games

EEG-based technology is widely used in serious game design since more wireless headsets that meet consumer criteria for wearability, price, portability, and ease-of-use are coming to the market. Originally, such technologies were mostly used in different medical applications, Brain Computer Interfaces (BCI) and neurofeedback games. The algorithms adopted in such applications are mainly based on power spectrum analysis, which may not be fully revealing the nonlinear complexity of the brain activities. In this paper, we first review neurofeedback games, EEG processing methods, and algorithms, and then propose a new nonlinear fractal dimension based approach to neurofeedback implementation targeting EEG-based serious games design. Only one channel is used in the proposed concentration quantification algorithm. The developed method was compared with other methods used for the concentration level recognition in neurofeedback games. The result analysis demonstrated an efficiency of the proposed approach. We designed and implemented new EEG-based 2D and 3D neurofeedback games that make the process of brain training more enjoyable.

Cybercampuses: design issues and future directions

In recent years, the usage of 3D cyberworlds for educational purposes has increased. The metaphors behind the visual design of such cyberworlds are quite diverse, from replication of real universities, art museums and scientific labs to non-existing fictitious places. In this paper, we focus specifically on “cybercampuses”, i.e. virtual worlds representing real educational institutions such as universities and schools. Based on the results of a case study that we have performed, this paper provides an initial set of requirements for a cybercampus representing an existing university. In this connection, we analyze place metaphors and associated design features of the Virtual Campus of Nanyang Technological University in Singapore, discuss the correspondence between the identified metaphors and associated educational goals, and provide directions for further development. Finally, we outline the major challenges for the future evolution of cybercampuses in the context of organizational, social and technological development.

Real-time EEG-based emotion recognition for music therapy

Music could change our emotions, could have an influence on our mood, and finally could affect our health. Music therapy is one of the oldest methods used for treating some diseases. Since music therapy is proved to be the helpful approach, we proposed to combine music therapy process with the real-time EEG-based human emotion recognition algorithm. By this, we could identify the user’s current emotional state, and based on such neurofeedback we could adjust the music therapy to the patient’s needs. The proposed emotion recognition algorithm could recognize in real-time six emotions such as fear, frustrated, sad, happy, pleasant, and satisfied. As the algorithm is based on an Arousal-Valence emotion model, it has a potential to recognize all emotions that could be defined by the 2-dimensional model. The experiments on emotion induction with sound stimuli from International Affective Digitized Sounds (IADS) database and with music stimuli and implemented questionnaire were proposed and realized. In this paper, we proposed a general EEG-enabled music therapy algorithm. It allows us to adapt the therapy to the predefined time of the treatment and adjust the music therapy session to the current emotional state of the user in a way as an experienced music therapist works.

Automatic clustering and boundary detection algorithm based on adaptive influence function

Clustering became a classical problem in databases, data warehouses, pattern recognition, artificial intelligence, and computer graphics. Applications in large spatial databases, point-based graphics, etc., give rise to new requirements for the clustering algorithms: automatic discovering of arbitrary shaped and/or non-homogeneous clusters, discovering of clusters located in low-dimensional hyperspace, detecting cluster boundaries. On that account, a new clustering and boundary detecting algorithm, ADACLUS, is proposed. It is based on the specially constructed adaptive influence function, and therefore, discovers clusters of arbitrary shapes and diverse densities, adequately captures clusters boundaries, and it is robust to noise. Normally ADACLUS performs clustering purely automatically without any preliminary parameter settings. But it also gives the user an optional possibility to set three parameters with clear meaning in order to adjust clustering for special applications. The algorithm was tested on various two-dimensional data sets, and it exhibited its effectiveness in discovering clusters of complex shapes and diverse densities. Linear complexity of the ADACLUS gives it an advantage over some well-known algorithms.

EEG Databases for Emotion Recognition

Emotion recognition from Electroencephalogram (EEG) rapidly gains interest from research community. Two affective EEG databases are presented in this paper. Two experiments are conducted to set up the databases. Audio and visual stimuli are used to evoke emotions during the experiments. The stimuli are selected from IADS and IAPS databases.14 subjects participated in each experiment. Emotiv EEG device is used for the data recording. The EEG data are rated by the participants with arousal, valence, and dominance levels. The correlation between powers of different EEG bands and the affective ratings is studied. The results agree with the literature findings and analyses of benchmark DEAP database that proves the reliability of the two databases. Similar brain patterns of emotions are obtained between the established databases and the benchmark database. A SVM-based emotion recognition algorithm is proposed and applied to both databases and the benchmark database. Use of a Fractal Dimension feature in combination with statistical and Higher Order Crossings (HOC) features gives us results with the best accuracy. Up to 8 emotions can be recognized. The accuracy is consistent between the established databases and the benchmark database.

Virtual orthopedic surgery training

Medicine is one of the most promising areas for emerging computer graphics and virtual reality techniques. VR training simulators let surgeons practice on virtual body tissue and get the same feedback they would experience in performing a real operation. Hybrid VR systems permit medical practitioners to view the patient overlaid with 3D data sets derived from 3D scanners, thus providing doctors and surgeons with pseudo X-ray vision. While currently available immersive VR surgery systems usually require expensive hardware and software, we developed a desktop VR orthopedic surgery training system that can run on commonly available personal computers.

Effective clustering and boundary detection algorithm based on Delaunay triangulation

In this paper, a new spatial clustering algorithm TRICLUST based on Delaunay triangulation is proposed. This algorithm treats clustering task by analyzing statistical features of data. For each data point, its values of statistical features are extracted from its neighborhood which effectively models the data proximity. By applying specifically built criteria function, TRICLUST is able to effectively handle data set with clusters of complex shapes and non-uniform densities, and with large amount of noises. One additional advantage of TRICLUST is the boundary detection function which is valuable for many real world applications such as geo-spatial data processing, point-based computer graphics, etc.

Real-Time Fractal-Based Valence Level Recognition from EEG

Emotions are important in human-computer interaction. Emotions could be classified based on 3-dimensional Valence-Arousal-Dominance model which allows defining any number of emotions even without discrete emotion labels. In this paper, we propose a real-time fractal dimension (FD) based valence level recognition algorithm from Electroencephalographic (EEG) signals. The FD-based feature is proposed as a valence dimension index in continuous emotion recognition. The thresholds are used to identify different levels of the valence dimension. The algorithm is tested on the EEG data labeled with different valence levels from the proposed and implemented experiment database and from the benchmark affective EEG database DEAP. The proposed algorithm is applied for recognition of 16 emotions defined by high/low arousal, high/low dominance and 4 levels of valence dimension. 9 levels of valence states with controlled dominance levels (high or low) can be recognized as well. The proposed algorithm can be implemented in different real-time applications such as emotional avatar and E-learning systems.