Omar AlZoubi

Classification of EEG for Affect Recognition: An Adaptive Approach

Research on affective computing is growing rapidly and new applications are being developed more frequently. They use information about the affective/mental states of users to adapt their interfaces or add new functionalities. Face activity, voice, text physiology and other information about the user are used as input to affect recognition modules, which are built as classification algorithms. Brain EEG signals have rarely been used to build such classifiers due to the lack of a clear theoretical framework. We present here an evaluation of three different classification techniques and their adaptive variations of a 10-class emotion recognition experiment. Our results show that affect recognition from EEG signals might be possible and an adaptive algorithm improves the performance of the classification task.

Detecting Naturalistic Expressions of Nonbasic Affect Using Physiological Signals

Signals from peripheral physiology (e.g., ECG, EMG, and GSR) in conjunction with machine learning techniques can be used for the automatic detection of affective states. The affect detector can be user-independent, where it is expected to generalize to novel users, or user-dependent, where it is tailored to a specific user. Previous studies have reported some success in detecting affect from physiological signals, but much of the work has focused on induced affect or acted expressions instead of contextually constrained spontaneous expressions of affect. This study addresses these issues by developing and evaluating user-independent and user-dependent physiology-based detectors of nonbasic affective states (e.g., boredom, confusion, curiosity) that were trained and validated on naturalistic data collected during interactions between 27 students and AutoTutor, an intelligent tutoring system with conversational dialogues. There is also no consensus on which techniques (i.e., feature selection or classification methods) work best for this type of data. Therefore, this study also evaluates the efficacy of affect detection using a host of feature selection and classification techniques on three physiological signals (ECG, EMG, and GSR) and their combinations. Two feature selection methods and nine classifiers were applied to the problem of recognizing eight affective states (boredom, confusion, curiosity, delight, flow/-engagement, surprise, and neutral). The results indicated that the user-independent modeling approach was not feasible; however, a mean kappa score of 0.25 was obtained for user-dependent models that discriminated among the most frequent emotions. The results also indicated that k-nearest neighbor and Linear Bayes Normal Classifier (LBNC) classifiers yielded the best affect detection rates. Single channel ECG, EMG, and GSR and three-channel multimodal models were generally more diagnostic than two--channel models.

Brain-computer interface: Next generation thought controlled distributed video game development platform

In this paper we present a system that uses the human ability to control a video game on a mobile device using electroencephalographic (EEG) Mu rhythms. The signals were obtained using a specially designed electrode cap and equipment, and sent through a Bluetooth connection to a PC that processes it in real time. The signal was then mapped onto two control signals and sent through wireless connection to a mobile gaming device BreakOut. We have also investigated the humans ability to play the video game by manipulating neuronal motor cortex activity in the presence of a visual feedback environment. The participants played the video game by using their thoughts only with up to 80% accuracy over controlling the target.

Affective modeling from multichannel physiology: analysis of day differences

Physiological signals are widely considered to contain affective information. Consequently, pattern recognition techniques such as classification are commonly used to detect affective states from physiological data. Previous studies have achieved some success in detecting affect from physiological measures, especially in controlled environments where emotions are experimentally induced. One challenge that arises is that physiological measures are expected to exhibit considerable day variations due to a number of extraneous factors such as environmental changes and sensor placements. These variations pose challenges to effectively classify affective sates from future physiological data; this is a common problem for real world requirements. The present study provides a quantitative analysis of day variations of physiological signals from different subjects. We propose a classifier ensemble approach using a Winnow algorithm to address the problem of day-variation in physiological signals. Our results show that the Winnow ensemble approach outperformed a static classification approach for detecting affective states from physiological signals that exhibited day variations.

Affect detection from non-stationary physiological data using ensemble classifiers

Affect detection from physiological signals has received considerable attention. One challenge is that physiological measures exhibit considerable variations over time, making classification of future data difficult. The present study addresses this issue by providing insights on how diagnostic physiological features of affect change over time. Affective physiological data (electrocardiogram, electromyogram, skin conductivity, and respiration) was collected from four participants over five sessions each. Classification performance of a number of training strategies, under different conditions of features selection and engineering, were compared using an adaptive classifier ensemble algorithm. Analysis of the performance of individual physiological channels for affect detection is also provided. The key result is that using pooled features set for affect detection is more accurate than using day-specific features. A decision fusion strategy which combines decisions from classifiers trained on individual channels data outperformed a features fusion strategy. Results also show that the performance of the ensemble is affected by the choice of the base classifier and the alpha factor used to update the member classifiers of the ensemble. Finally, the corrugator and zygomatic facial EMGs were found to be more reliable measures for detecting the valence component of affect compared to other channels.

A Scalable Intelligent Tutoring System Framework for Computer Science Education

Computer Science is a difficult subject with many fundamentals to be taught, usually involving a steep learning curve for many students. It is some of these initial challenges that can turn students away from computer science. We have been developing a new Intelligent Tutoring System, ChiQat-Tutor, that focuses on tutoring of Computer Science fundamentals. Here, we outline the system under development, while bringing particular attention to its architecture and how it attains the primary goals of being easily extensible and providing a low barrier of entry to the end user. The system is broadly broken down into lessons, teaching strategies, and utilities, which work together to promote seamless integration of components. We also cover currently developed components in the form of a case study, as well as detailing our experience of deploying it to an undergraduate Computer Science classroom, leading to learning gains on par with prior work.

Worked Out Examples in Computer Science Tutoring

We annotated and analyzed Worked Out Examples (WOEs) in a corpus of tutoring dialogues on Computer Science data structures. We found that some dialogue moves that occur within WOEs, or se- quences thereof, correlate with learning. Features of WOEs such as length also correlate with learning for some data structures. These re- sults will be used to augment the tutorial tactics available to iList, an ITS that helps student learn linked lists.

Worked-out Examples in a Computer Science Intelligent Tutoring System

Our CS Intelligent Tutoring System (ITS), ChiQat-Tutor, aims at aiding students in overcoming the initial difficulties in CS education, such as learning data structures. Here, we show our work on utilizing Worked-out Examples (WOE) in our linked list lesson. Despite being a promising strategy, we find that it can be detrimental to student growth.

A Hybrid Model for Teaching Recursion

Novice programmers struggle to understand the concept of recursion, partly because of unfamiliarity with recursive activities, difficulty with visualizing program execution, and difficulty understanding its back flow of control. In this paper we discuss the conceptual and program visualization approaches to teaching recursion. We also introduce our approach to teaching recursion in the ChiQat-Tutor system that relies on ideas from both approaches. ChiQat-Tutor will help Computer Science students learn recursion, develop accurate mental models of recursion, and serve as an effective visualization tool with which hidden contexts of recursion can become evident.

Siento: an experimental platform for behavior and psychophysiology in HCI

We describe Siento, a system to perform different types of affective computing studies. The platform allows for dimensional or categorical models of emotions, self-reported vs. third party reporting and can record and process multiple types of modalities including video, physiology and text. It has been used already in a number of studies. This type of systems can improve the repeatability of experiments. The system is also used for data acquisition, feature extraction and data analysis applying machine learning techniques.