Minh Khoa Nguyen

Real-Time EEG-Based Human Emotion Recognition and Visualization

Emotions accompany everyone in the daily life, playing a key role in non-verbal communication, and they are essential to the understanding of human behavior. Emotion recognition could be done from the text, speech, facial expression or gesture. In this paper, we concentrate on recognition of “inner” emotions from electroencephalogram (EEG) signals as humans could control their facial expressions or vocal intonation. The need and importance of the automatic emotion recognition from EEG signals has grown with increasing role of brain computer interface applications and development of new forms of human-centric and human-driven interaction with digital media. We propose fractal dimension based algorithm of quantification of basic emotions and describe its implementation as a feedback in 3D virtual environments. The user emotions are recognized and visualized in real time on his/her avatar adding one more so-called “emotion dimension” to human computer interfaces.

EEG-Based "Serious" Games Design for Medical Applications

Recently, EEG-based technology has become more popular in “serious” games designs and developments since new wireless headsets that meet consumer demand for wear ability, price, portability and ease-of-use are coming to the market. Originally, EEG-based technologies were used in neurofeedback games and brain-computer interfaces. Now, such technologies could be used in entertainment, e-learning and new medical applications. In this paper, we review on neurofeedback game designs and algorithms, and propose design, algorithm, and implementation of new EEG-based 2D and 3D concentration games. Possible future medical applications of the games are discussed.

Fractal dimension based neurofeedback in serious games

EEG-based technology is widely used in serious game design since more wireless headsets that meet consumer criteria for wearability, price, portability, and ease-of-use are coming to the market. Originally, such technologies were mostly used in different medical applications, Brain Computer Interfaces (BCI) and neurofeedback games. The algorithms adopted in such applications are mainly based on power spectrum analysis, which may not be fully revealing the nonlinear complexity of the brain activities. In this paper, we first review neurofeedback games, EEG processing methods, and algorithms, and then propose a new nonlinear fractal dimension based approach to neurofeedback implementation targeting EEG-based serious games design. Only one channel is used in the proposed concentration quantification algorithm. The developed method was compared with other methods used for the concentration level recognition in neurofeedback games. The result analysis demonstrated an efficiency of the proposed approach. We designed and implemented new EEG-based 2D and 3D neurofeedback games that make the process of brain training more enjoyable.

Real-time EEG-based emotion recognition for music therapy

Music could change our emotions, could have an influence on our mood, and finally could affect our health. Music therapy is one of the oldest methods used for treating some diseases. Since music therapy is proved to be the helpful approach, we proposed to combine music therapy process with the real-time EEG-based human emotion recognition algorithm. By this, we could identify the user’s current emotional state, and based on such neurofeedback we could adjust the music therapy to the patient’s needs. The proposed emotion recognition algorithm could recognize in real-time six emotions such as fear, frustrated, sad, happy, pleasant, and satisfied. As the algorithm is based on an Arousal-Valence emotion model, it has a potential to recognize all emotions that could be defined by the 2-dimensional model. The experiments on emotion induction with sound stimuli from International Affective Digitized Sounds (IADS) database and with music stimuli and implemented questionnaire were proposed and realized. In this paper, we proposed a general EEG-enabled music therapy algorithm. It allows us to adapt the therapy to the predefined time of the treatment and adjust the music therapy session to the current emotional state of the user in a way as an experienced music therapist works.

Fractal-Based Brain State Recognition from EEG in Human Computer Interaction

Real-time brain states recognition from Electroencephalogram (EEG) could add a new dimension in an immersive human-computer interaction. As EEG signal is considered to have a fractal nature, we proposed and developed a general fractal based spatio-temporal approach to brain states recognition including the concentration level, stress level, and emotion recognition. Our hypothesis is that changes of fractal dimension values of EEG over time correspond to the brain states changes. Overall brain state recognition algorithms were proposed and described. Fractal dimension values were calculated by the implemented Higuchi and Box-counting methods. Real-time subject-dependent classification algorithms based on threshold FD values calculated during a short training session were proposed and implemented. Based on the proposed real-time algorithms, neurofeedback games for concentration and stress management training such as “Brain Chi”, “Dancing Robot”, “Escape”, and “Apples”, and emotion-enabled applications such as emotion-enabled avatar, music therapy, and emotion-based search were designed and implemented.

Fractal-based approach in analysis of intracranial pressure (ICP) in severe head injury

Physiologic monitoring is a key to guiding severe head injury therapy in the neurological intensive care unit (ICU), of which intracranial pressure (ICP) plays a critical role. Currently, decision-making with respect to escalating therapy in the stepwise protocol for refractory raised ICP is made by interpreting absolute values of the physiologic parameters coupled with the doctor clinician knowledge and experience. In this paper, we proposed a fractal dimension-based method for processing and quantification of time-series ICP data for neurological monitoring to predict the transition from maximal medical therapy to decompressive craniectomy. The fractal analysis results in a quantitative measure, known as a fractal dimension (FD), describing the self-similar patterns observed in time-series data. In this paper, we proposed fractal based method to analyse ICP data to predict changes in the patient state. We processed ICP data of 9 patients before and after decompression with well known Box-counting and Higuchi algorithms. Our results suggest that FD values could be used as a valuable additional parameter to indicate the need for surgical decompression. The FD based method could potentially be implemented as a software tool in intensive care units.

EEG-Enabled Human–Computer Interaction and Applications

Modern Electroencephalogram (EEG) devices and recent advances in signal processing and pattern recognition algorithms have made it possible to integrate brain state recognition algorithms in human–computer interfaces. These systems could allow the user to input some information into the computer system just by “thinking.” Brain states such as stress level, concentration level, human emotions, etc. could be recognized in real-time from EEG. In this work, we describe a general approach to EEG-enabled human–computer interaction and propose applications based on the real-time brain state recognition. When the user receives stimuli such as visual, audio, haptic, etc. from the computer system, and the information is processed in his/her brain, some brain states such as alertness level, concentration level or even human emotions could be recognized from his/her EEG in real-time. Then, the command to the system is formed based on the recognition results. Depending on the application, the command could be the recognized emotion, concentration level, pain level, etc. Real-time EEG-enabled systems could be developed for medical applications, performance improvement, entertainment, or even neuromarketing. In this work, software systems for medical applications and entertainment based on emotion recognition and concentration level recognition algorithms are described.