Danny Plass-Oude Bos

Turning shortcomings into challenges: Brain–computer interfaces for games

In recent years we have seen a rising interest in brain-computer interfacing for human-computer interaction and potential game applications. Until now, however, we have almost only seen attempts where BCI is used to measure the affective state of the user or in neurofeeedback games. There have hardly been any attempts to design BCI games where BCI is considered to be one of the possible input modalities that can be used to control the game. One reason may be that research still follows the paradigms of the traditional, medically oriented, BCI approaches. In this paper we discuss current BCI research from the viewpoint of games and game design. It is hoped that this survey will make clear that we need to design different games than we used to, but that such games can nevertheless be interesting and exciting.

Brain-Computer Interfacing and Games

Recently research into Brain-Computer Interfacing (BCI) applications for healthy users, such as games, has been initiated. But why would a healthy person use a still-unproven technology such as BCI for game interaction? BCI provides a combination of information and features that no other input modality can offer. But for general acceptance of this technology, usability and user experience will need to be taken into account when designing such systems. Therefore, this chapter gives an overview of the state of the art of BCI in games and discusses the consequences of applying knowledge from Human-Computer Interaction (HCI) to the design of BCI for games. The integration of HCI with BCI is illustrated by research examples and showcases, intended to take this promising technology out of the lab. Future research needs to move beyond feasibility tests, to prove that BCI is also applicable in realistic, real-world settings.

Experiencing BCI Control in a Popular Computer Game

Brain-computer interfaces (BCIs) are not only being developed to aid disabled individuals with motor substitution, motor recovery, and novel communication possibilities, but also as a modality for healthy users in entertainment and gaming. This study investigates whether the incorporation of a BCI in the popular game World of Warcraft (WoW) has effects on the user experience. A BCI control channel based on parietal alpha band power is used to control the shape and function of the avatar in the game. In the experiment, participants (n=42) , a mix of experienced and inexperienced WoW players, played with and without the use of BCI in a within-subjects design. Participants themselves could indicate when they wanted to stop playing. Actual and estimated duration was recorded and questionnaires on presence and control were administered. Afterwards, oral interviews were taken. No difference in actual duration was found between conditions. Results indicate that the difference between estimated and actual duration was not related to user experience but was person specific. When using a BCI, control and involvement were rated lower. But BCI control did not significantly decrease fun. During interviews, experienced players stated that they saw potential in the application of BCIs in games with complex interfaces such as WoW. This study suggests that BCI as an additional control can be as much fun and natural to use as keyboard/mouse control, even if the amount of control is limited.

A study in user-centered design and evaluation of mental tasks for BCI

Current brain-computer interfacing (BCI) research focuses on detection performance, speed, and bit rates. However, this is only a small part of what is important to the user. From human-computer interaction (HCI) research, we can apply the paradigms of user-centered design and evaluation, to improve the usability and user experience. Involving the users in the design process may also help in moving beyond the limited mental tasks that are currently common in BCI systems. To illustrate the usefulness of these methods to BCI, we involved potential users in the design process of a BCI system, resulting in three new mental tasks. The experience of using these mental tasks was then evaluated within a prototype BCI system using a commercial online role-playing game. Results indicate that user preference for certain mental tasks is primarily based on the recognition of brain activity by the system, and secondly on the ease of executing the task.

How Much Control Is Enough? Influence of Unreliable Input on User Experience

Brain-computer interfaces (BCI) provide a valuable new input modality within human-computer interaction systems. However, like other body-based inputs such as gesture or gaze based systems, the system recognition of input commands is still far from perfect. This raises important questions, such as what level of control should such an interface be able to provide. What is the relationship between actual and perceived control? And in the case of applications for entertainment in which fun is an important part of user experience, should we even aim for the highest level of control, or is the optimum elsewhere? In this paper, we evaluate whether we can modulate the amount of control and if a game can be fun with less than perfect control. In the experiment users (n=158) played a simple game in which a hamster has to be guided to the exit of a maze. The amount of control the user has over the hamster is varied. The variation of control through confusion matrices makes it possible to simulate the experience of using a BCI, while using the traditional keyboard for input. After each session the user completed a short questionnaire on user experience and perceived control. Analysis of the data showed that the perceived control of the user could largely be explained by the amount of control in the respective session. As expected, user frustration decreases with increasing control. Moreover, the results indicate that the relation between fun and control is not linear. Although at lower levels of control fun does increase with improved control, the level of fun drops just before perfect control is reached (with an optimum around 96%). This poses new insights for developers of games who want to incorporate some form of BCI or other modality with unreliable input in their game: for creating a fun game, unreliable input can be used to create a challenge for the user.

Evaluating User Experience of Actual and Imagined Movement in BCI Gaming

Most research on Brain-Computer Interfaces BCI focuses on developing ways of expression for disabled people who are not able to communicate through other means. Recently it has been shown that BCI can also be used in games to give users a richer experience and new ways to interact with a computer or game console. This paper describes research conducted to find out what the differences are between using actual and imagined movement as modalities in a BCI game. Results show that there are significant differences in user experience and that actual movement is a more robust way of communicating through a BCI.

Social Interaction in a Cooperative Brain-computer Interface Game

Does using a brain-computer interface (BCI) influence the social interaction between people when playing a cooperative game? By measuring the amount of speech, utterances, instrumental gestures and empathic gestures during a cooperative game where two participants had to reach a certain goal, and questioning participants about their own experience afterwards this study attempts to provide answers to this question. The results showed that social interaction changed when using a BCI compared to using a mouse. There was a higher amount of utterances and empathic gestures. This indicates that the participants reacted more to the higher difficulty of the BCI selection method. Participants also reported that they felt they cooperated better during the use of the mouse.

Perspectives on user experience evaluation of brain-computer interfaces

The research on brain-computer interfaces (BCIs) is pushing hard to bring technologies out of the lab and into society and onto the market. The nascent merge between the field of BCI and humancomputer interaction (HCI) is paving the way for new applications such as BCI-controlled gaming. The evaluation or success of BCI technologies is often based on how accurate the control of a user is with the technology. However, while this is still key to its usability, other factors that influence the user experience (UX) can make or break a technology. In this paper we first review studies which investigated user experience with BCIs. Second, we will discuss how HCI approaches can contribute to the evaluation of BCIs. Finally, we propose to develop a standardized questionnaire for evaluating BCIs for entertainment purposes.

Measuring immersion and affect in a brain-computer interface game

Brain-computer interfaces (BCIs) have widely been used in medical applications, to facilitate making selections. However, whether they are suitable for recreational applications is unclear as they have rarely been evaluated for user experience. As the scope of the BCI applications is expanding from medical to recreational use, the expectations of BCIs are also changing. Although the performance of BCIs is still important, finding suitable BCI modalities and investigating their influence on user experience demand more and more attention. In this study a BCI selection method and a comparable non-BCI selection method were integrated into a computer game to evaluate user experience in terms of immersion and affect. An experiment with seventeen participants showed that the BCI selection method was more immersive and positively affective than the non-BCI selection method. Participants also seemed to be more indulgent towards the BCI selection method.

Brain–Computer Interfaces and User Experience Evaluation

The research on brain–computer interfaces (BCIs) is pushing hard to bring technologies out of the lab, into society and onto the market. The newly developing merge of the field of BCI with human–computer interaction (HCI) is paving the way for new applications such as BCI-controlled games. The evaluation or success of BCI technologies is often based on how accurate the control of a user is over the technology. However, while this is still key to its usability, other factors that influence the user experience (UX) can make or break a technology. In this paper we first review studies that investigated user experience with BCIs. Second, we will discuss how methods from the field of HCI can contribute to the evaluation of BCIs. From experience drawn from two case studies we provide recommendations for evaluating BCIs.