Gabor Aranyi

Subliminal cueing of selection behavior in a virtual environment

The performance of current graphics engines makes it possible to incorporate subliminal cues within virtual environments (VEs), providing an additional way of communication, fully integrated with the exploration of a virtual scene. In order to advance the application of subliminal information in this area, it is necessary to explore in the psychological literature how techniques previously reported as rendering information subliminal can be successfully implemented in VEs. Previous literature has also described the effects of subliminal cues as quantitatively modest, which raises the issue of their inclusion in practical tasks. We used a 3D rendering engine (Unity3D) to implement a masking paradigm within the context of a realistic scene and a familiar (kitchen) environment. We report significant effects of subliminal cueing on the selection of objects in a virtual scene, demonstrating the feasibility of subliminal cueing in VEs. Furthermore, we show that multiple iterations of masked objects within a trial, as well as the speeding of selection choices, can substantially reinforce the impact of subliminal cues. This is consistent with previous findings suggesting that the effect of subliminal stimuli fades rapidly. We conclude by proposing, as part of further work, possible mechanisms for the inclusion of subliminal cueing in intelligent interfaces to maximize their effects.

Covert Persuasive Technologies: Bringing Subliminal Cues to Human-Computer Interaction

The capability of machines to covertly persuade humans is both exciting and ethically concerning. In the present study we aim to bring subliminal masked stimulus paradigms to realistic environments, through Virtual Environments. The goal is to test if such paradigms are applicable to realistic setups while identifying the major challenges when doing so. We designed a study in which the user performed a realistic selection task in a virtual kitchen. For trials below one-second reaction time, we report significant effect of subliminal cues on the selection behavior. We conclude the study with a discussion of the challenges of bringing subliminal cueing paradigms to realistic HCI setups. Ethical concerns when designing covertly persuasive systems are discussed as well.

Affective Interaction with a Virtual Character Through an fNIRS Brain-Computer Interface.

Affective brain-computer interfaces (BCI) harness Neuroscience knowledge to develop affective interaction from first principles. In this article, we explore affective engagement with a virtual agent through Neurofeedback (NF). We report an experiment where subjects engage with a virtual agent by expressing positive attitudes towards her under a NF paradigm. We use for affective input the asymmetric activity in the dorsolateral prefrontal cortex (DL-PFC), which has been previously found to be related to the high-level affective-motivational dimension of approach/avoidance. The magnitude of left-asymmetric DL-PFC activity, measured using functional near infrared spectroscopy (fNIRS) and treated as a proxy for approach, is mapped onto a control mechanism for the virtual agent’s facial expressions, in which action units (AUs) are activated through a neural network. We carried out an experiment with 18 subjects, which demonstrated that subjects are able to successfully engage with the virtual agent by controlling their mental disposition through NF, and that they perceived the agent’s responses as realistic and consistent with their projected mental disposition. This interaction paradigm is particularly relevant in the case of affective BCI as it facilitates the volitional activation of specific areas normally not under conscious control. Overall, our contribution reconciles a model of affect derived from brain metabolic data with an ecologically valid, yet computationally controllable, virtual affective communication environment.

Towards emotional regulation through neurofeedback

This paper discusses the potential of Brain-Computer Interfaces based on neurofeedback methods to support emotional control and pursue the goal of emotional control as a mechanism for human augmentation in specific contexts. We illustrate this discussion through two proof-of-concept, fully-implemented experiments: one controlling disposition towards virtual characters using pre-frontal alpha asymmetry, and the other aimed at controlling arousal through activity of the amygdala. In the first instance, these systems are intended to explore augmentation technologies that would be incorporated into various media-based systems rather than permanently affect user behaviour.

Anger-based BCI Using fNIRS Neurofeedback

Functional near-infrared spectroscopy (fNIRS) holds increasing potential for Brain-Computer Interfaces (BCI) due to its portability, ease of application, robustness to movement artifacts, and relatively low cost. The use of fNIRS to support the development of affective BCI has received comparatively less attention, despite the role played by the prefrontal cortex in affective control, and the appropriateness of fNIRS to measure prefrontal activity. We present an active, fNIRS-based neurofeedback (NF) interface, which uses differential changes in oxygenation between the left and right sides of the dorsolateral prefrontal cortex to operationalize BCI input. The system is activated by users generating a state of anger, which has been previously linked to increased left prefrontal asymmetry. We have incorporated this NF interface into an experimental platform adapted from a virtual 3D narrative, in which users can express anger at a virtual character perceived as evil, causing the character to disappear progressively. Eleven subjects used the system and were able to successfully perform NF despite minimal training. Extensive analysis confirms that success was associated with the intent to express anger. This has positive implications for the design of affective BCI based on prefrontal asymmetry.

3-D route-planning support for navigation in a complex indoor environment

Three-dimensional (3-D) route-planning support offers a promising solution to overcome problems with wayfinding in complex indoor environments. An experiment was conducted to test the effect of 3-D route-planning support in a realistic setting, a large hospital building, during normal operation. Forty participants performed navigation tasks either with (n = 20) or without (n = 20) 3-D route-planning support. Support resulted in faster navigation, more use of artwork specifically installed to aid wayfinding, fewer navigation errors, less disorientation and less anxiety. In addition, participants used different strategies for wayfinding: without navigation support they used signs and route colour, but with navigation support they used not only the artwork, but also the existing furniture and other landmarks. The acceptance of 3-D route-planning support was high. Overall, the results support the value of 3-D route-planning support.

BCI Control of Heuristic Search Algorithms

The ability to develop Brain-Computer Interfaces (BCI) to Intelligent Systems would offer new perspectives in terms of human supervision of complex Artificial Intelligence (AI) systems, as well as supporting new types of applications. In this article, we introduce a basic mechanism for the control of heuristic search through fNIRS-based BCI. The rationale is that heuristic search is not only a basic AI mechanism but also one still at the heart of many different AI systems. We investigate how users’ mental disposition can be harnessed to influence the performance of heuristic search algorithm through a mechanism of precision-complexity exchange. From a system perspective, we use weighted variants of the A* algorithm which have an ability to provide faster, albeit suboptimal solutions. We use recent results in affective BCI to capture a BCI signal, which is indicative of a compatible mental disposition in the user. It has been established that Prefrontal Cortex (PFC) asymmetry is strongly correlated to motivational dispositions and results anticipation, such as approach or even risk-taking, and that this asymmetry is amenable to Neurofeedback (NF) control. Since PFC asymmetry is accessible through fNIRS, we designed a BCI paradigm in which users vary their PFC asymmetry through NF during heuristic search tasks, resulting in faster solutions. This is achieved through mapping the PFC asymmetry value onto the dynamic weighting parameter of the weighted A* (WA*) algorithm. We illustrate this approach through two different experiments, one based on solving 8-puzzle configurations, and the other on path planning. In both experiments, subjects were able to speed up the computation of a solution through a reduction of search space in WA*. Our results establish the ability of subjects to intervene in heuristic search progression, with effects which are commensurate to their control of PFC asymmetry: this opens the way to new mechanisms for the implementation of hybrid cognitive systems.

Using fNIRS for Prefrontal-Asymmetry Neurofeedback: Methods and Challenges

Functional near-infrared spectroscopy (fNIRS) has become increasingly accessible in recent years, which allows this relatively low-cost and portable brain sensing modality for the application of brain-computer interfaces (BCI). Although there is a growing body of research on fNIRS-based BCI utilising users’ covert psychophysiological activity, there is comparably less research on active BCI, where users engage in thinking strategies with the explicit intention of controlling the behaviour of an interactive system. We draw on four empirical studies, where participants received real-time neurofeedback (NF) of left-asymmetric increase in activation in their dorsolateral prefrontal cortex (DL-PFC), which has previously been identified as a correlate of approach-related motivational tendencies. We discuss methodological considerations and challenges, and provide recommendations about brain-signal selection and integration, NF protocol design, post-hoc and real-time applications of NF success criteria, continuous visual feedback, and individualised feedback based on the variations of the brain-signal in a reference condition.

Integrating virtual agents in BCI neurofeedback systems

The recent development of Brain-Computer Interfaces (BCI) to Virtual World has resulted in a growing interest in realistic visual feedback. In this paper, we investigate the potential role of Virtual Agents in neurofeedback systems, which constitute an important paradigm for BCI. We discuss the potential impact of virtual agents on some important determinants of neurofeedback in the context of affective BCI. Throughout the paper, we illustrate our presentation with two fully implemented neurofeedback prototypes featuring virtual agents: the first is an interactive narrative in which the user empathises with the character through neurofeedback; the second recreates a natural environment in which crowd behaviour becomes a metaphor for arousal and the user engages in emotional regulation.