Mannes Poel

Brain-Computer Interfacing and Games

Recently research into Brain-Computer Interfacing (BCI) applications for healthy users, such as games, has been initiated. But why would a healthy person use a still-unproven technology such as BCI for game interaction? BCI provides a combination of information and features that no other input modality can offer. But for general acceptance of this technology, usability and user experience will need to be taken into account when designing such systems. Therefore, this chapter gives an overview of the state of the art of BCI in games and discusses the consequences of applying knowledge from Human-Computer Interaction (HCI) to the design of BCI for games. The integration of HCI with BCI is illustrated by research examples and showcases, intended to take this promising technology out of the lab. Future research needs to move beyond feasibility tests, to prove that BCI is also applicable in realistic, real-world settings.

Action systems and action refinement in the development of parallel systems

A new notion of refinement and several other new operators are proposed that allow for a compositional algebraic characterization of action systems and serializability in distributed database systems. A simple design language is introduced and is provided with a semantics essentially based on partial order models.

Generation of Facial Expressions from Emotion Using a Fuzzy Rule Based System

We propose a fuzzy rule-based system to map representations of the emotional state of an animated agent onto muscle contraction values for the appropriate facial expressions. Our implementation pays special attention to the way in which continuous changes in the intensity of emotions can be displayed smoothly on the graphical face. The rule system we have defined implements the patterns described by psychologists and researchers dealing with facial expressions of humans, including rules for displaying blends of expressions.

Comparison of silhouette shape descriptors for example-based human pose recovery

Automatically recovering human poses from visual input is useful but challenging due to variations in image space and the high dimensionality of the pose space. In this paper, we assume that a human silhouette can be extracted from monocular visual input. We compare three shape descriptors that are used in the encoding of silhouettes: Fourier descriptors, shape contexts and Hu moments. An example-based approach is taken to recover upper body poses from these descriptors. We perform experiments with deformed silhouettes to test each descriptors robustness against variations in body dimensions, viewpoint and noise. It is shown that Fourier descriptors and shape context histograms outperform Hu moments for all deformations

ParleE: An Adaptive Plan Based Event Appraisal Model of Emotions

We propose ParleE, a quantitative, flexible and adaptive model of emotions for a conversational agent in a multi-agent environment capable of multimodal communication. ParleE appraises events based on learning and a probabilistic planning algorithm. ParleE also models personality and motivational states and their role in determining the way the agent experiences emotion.

Affective Pacman: A Frustrating Game for Brain-Computer Interface Experiments

We present the design and development of Affective Pacman, a game that induces frustration to study the effect of user state changes on the EEG signal. Affective Pacman is designed to induce frustration for short periods, and allows the synchronous recording of a wide range of sensors, such as physiological sensors and EEG in addition to the game state. A self-assessment is integrated in the game to track changes in user state. Preliminary results indicate a signicant effect of the frustration induction on the EEG.

Experiencing BCI Control in a Popular Computer Game

Brain-computer interfaces (BCIs) are not only being developed to aid disabled individuals with motor substitution, motor recovery, and novel communication possibilities, but also as a modality for healthy users in entertainment and gaming. This study investigates whether the incorporation of a BCI in the popular game World of Warcraft (WoW) has effects on the user experience. A BCI control channel based on parietal alpha band power is used to control the shape and function of the avatar in the game. In the experiment, participants (n=42) , a mix of experienced and inexperienced WoW players, played with and without the use of BCI in a within-subjects design. Participants themselves could indicate when they wanted to stop playing. Actual and estimated duration was recorded and questionnaires on presence and control were administered. Afterwards, oral interviews were taken. No difference in actual duration was found between conditions. Results indicate that the difference between estimated and actual duration was not related to user experience but was person specific. When using a BCI, control and involvement were rated lower. But BCI control did not significantly decrease fun. During interviews, experienced players stated that they saw potential in the application of BCIs in games with complex interfaces such as WoW. This study suggests that BCI as an additional control can be as much fun and natural to use as keyboard/mouse control, even if the amount of control is limited.

Valence, arousal and dominance in the EEG during game play

In this paper, we describe our investigation of traces of naturally occurring emotions in electrical brain signals, that can be used to build interfaces that respond to our emotional state. This study confirms a number of known affective correlates in a realistic, uncontrolled environment for the emotions of valence (or pleasure), arousal and dominance: (1) a significant decrease in frontal power in the theta range is found for increasingly positive valence, (2) a significant frontal increase in power in the alpha range is associated with increasing emotional arousal, (3) a significant right posterior power increase in the delta range correlates with increasing arousal and (4) asymmetry in power in the lower alpha bands correlates with self-reported valence. Furthermore, asymmetry in the higher alpha bands correlates with self-reported dominance. These last two effects provide a simple measure for subjective feelings of pleasure and feelings of control.

A tractable hybrid ddn–pomdp approach to affective dialogue modeling for probabilistic frame-based dialogue systems

We propose a novel approach to developing a tractable affective dialogue model for probabilistic frame-based dialogue systems. The affective dialogue model, based on Partially Observable Markov Decision Process (POMDP) and Dynamic Decision Network (DDN) techniques, is composed of two main parts: the slot-level dialogue manager and the global dialogue manager. It has two new features: (1) being able to deal with a large number of slots and (2) being able to take into account some aspects of the users affective state in deriving the adaptive dialogue strategies. Our implemented prototype dialogue manager can handle hundreds of slots, where each individual slot might have hundreds of values. Our approach is illustrated through a route navigation example in the crisis management domain. We conducted various experiments to evaluate our approach and to compare it with approximate POMDP techniques and handcrafted policies. The experimental results showed that the DDN–POMDP policy outperforms three handcrafted policies when the users action error is induced by stress as well as when the observation error increases. Further, performance of the one-step look-ahead DDN–POMDP policy after optimizing its internal reward is close to state-of-the-art approximate POMDP counterparts.

A study in user-centered design and evaluation of mental tasks for BCI

Current brain-computer interfacing (BCI) research focuses on detection performance, speed, and bit rates. However, this is only a small part of what is important to the user. From human-computer interaction (HCI) research, we can apply the paradigms of user-centered design and evaluation, to improve the usability and user experience. Involving the users in the design process may also help in moving beyond the limited mental tasks that are currently common in BCI systems. To illustrate the usefulness of these methods to BCI, we involved potential users in the design process of a BCI system, resulting in three new mental tasks. The experience of using these mental tasks was then evaluated within a prototype BCI system using a commercial online role-playing game. Results indicate that user preference for certain mental tasks is primarily based on the recognition of brain activity by the system, and secondly on the ease of executing the task.