Bruno Herbelin

Visual capture and the experience of having two bodies – Evidence from two different virtual reality techniques

In neurology and psychiatry the detailed study of illusory own body perceptions has suggested close links between bodily processing and self-consciousness. One such illusory own body perception is heautoscopy where patients have the sensation of being reduplicated and to exist at two or even more locations. In previous experiments, using a video head-mounted display, self-location and self-identification were manipulated by applying conflicting visuo-tactile information. Yet the experienced singularity of the self was not affected, i.e., participants did not experience having multiple bodies or selves. In two experiments presented in this paper, we investigated self-location and self-identification while participants saw two virtual bodies (video-generated in study 1 and 3D computer generated in study 2) that were stroked either synchronously or asynchronously with their own body. In both experiments, we report that self-identification with two virtual bodies was stronger during synchronous stroking. Furthermore, in the video generated setup with synchronous stroking participants reported a greater feeling of having multiple bodies than in the control conditions. In study 1, but not in study 2, we report that self-location – measured by anterior posterior drift – was significantly shifted towards the two bodies in the synchronous condition only. Self-identification with two bodies, the sensation of having multiple bodies, and the changes in self-location show that the experienced singularity of the self can be studied experimentally. We discuss our data with respect to ownership for supernumerary hands and heautoscopy. We finally compare the effects of the video and 3D computer generated head-mounted display technology and discuss the possible benefits of using either technology to induce changes in illusory self-identification with a virtual body.

Turning Body and Self Inside Out Visualized Heartbeats Alter Bodily Self-Consciousness and Tactile Perception

Prominent theories highlight the importance of bodily perception for self-consciousness, but it is currently not known whether bodily perception is based on interoceptive or exteroceptive signals or on integrated signals from these anatomically distinct systems. In the research reported here, we combined both types of signals by surreptitiously providing participants with visual exteroceptive information about their heartbeat: A real-time video image of a periodically illuminated silhouette outlined participants’ (projected, “virtual”) bodies and flashed in synchrony with their heartbeats. We investigated whether these “cardio-visual” signals could modulate bodily self-consciousness and tactile perception. We report two main findings. First, synchronous cardio-visual signals increased self-identification with and self-location toward the virtual body, and second, they altered the perception of tactile stimuli applied to participants’ backs so that touch was mislocalized toward the virtual body. We argue that the integration of signals from the inside and the outside of the human body is a fundamental neurobiological process underlying self-consciousness.

Immersive VR decision training: telling interactive stories featuring advanced virtual human simulation technologies

Based on the premise of a synergy between the interactive storytelling and VR training simulation this paper treats the main issues involved in practical realization of an immersive VR decision training system supporting possibly broad spectrum of scenarios featuring interactive virtual humans. The paper describes a concrete concept of such a system and its practical realization example.

Right insular damage decreases heartbeat awareness and alters cardio- visual effects on bodily self-consciousness

Recent evidence suggests that multisensory integration of bodily signals involving exteroceptive and interoceptive information modulates bodily aspects of self-consciousness such as self-identification and self-location. In the so-called Full Body Illusion subjects watch a virtual body being stroked while they perceive tactile stimulation on their own body inducing illusory self-identification with the virtual body and a change in self-location towards the virtual body. In a related illusion, it has recently been shown that similar changes in self-identification and self-location can be observed when an interoceptive signal is used in association with visual stimulation of the virtual body (i.e., participants observe a virtual body illuminated in synchrony with their heartbeat). Although brain imaging and neuropsychological evidence suggest that the insular cortex is a core region for interoceptive processing (such as cardiac perception and awareness) as well as for self-consciousness, it is currently not known whether the insula mediates cardio-visual modulation of self-consciousness. Here we tested the involvement of insular cortex in heartbeat awareness and cardio-visual manipulation of bodily self-consciousness in a patient before and after resection of a selective right neoplastic insular lesion. Cardio-visual stimulation induced an abnormally enhanced state of bodily self-consciousness; in addition, cardio-visual manipulation was associated with an experienced loss of the spatial unity of the self (illusory bi-location and duplication of his body), not observed in healthy subjects. Heartbeat awareness was found to decrease after insular resection. Based on these data we propose that the insula mediates interoceptive awareness as well as cardio-visual effects on bodily self-consciousness and that insular processing of interoceptive signals is an important mechanism for the experienced unity of the self.

Balancing awareness: Vestibular signals modulate visual consciousness in the absence of awareness

The processing of visual and vestibular information is crucial for perceiving self-motion. Visual cues, such as optic flow, have been shown to induce and alter vestibular percepts, yet the role of vestibular information in shaping visual awareness remains unclear. Here we investigated if vestibular signals influence the access to awareness of invisible visual signals. Using natural vestibular stimulation (passive yaw rotations) on a vestibular self-motion platform, and optic flow masked through continuous flash suppression (CFS) we tested if congruent visual-vestibular information would break interocular suppression more rapidly than incongruent information. We found that when the unseen optic flow was congruent with the vestibular signals perceptual suppression as quantified with the CFS paradigm was broken more rapidly than when it was incongruent. We argue that vestibular signals impact the formation of visual awareness through enhanced access to awareness for congruent multisensory stimulation.

Vestibular-somatosensory interactions: effects of passive whole-body rotation on somatosensory detection.

Vestibular signals are strongly integrated with information from several other sensory modalities. For example, vestibular stimulation was reported to improve tactile detection. However, this improvement could reflect either a multimodal interaction or an indirect interaction driven by vestibular effects on spatial attention and orienting. Here we investigate whether natural vestibular activation induced by passive whole-body rotation influences tactile detection. In particular, we assessed the ability to detect faint tactile stimuli to the fingertips of the left and right hand during spatially congruent or incongruent rotations. We found that passive whole-body rotations significantly enhanced sensitivity to faint shocks, without affecting response bias. Critically, this enhancement of somatosensory sensitivity did not depend on the spatial congruency between the direction of rotation and the hand stimulated. Thus, our results support a multimodal interaction, likely in brain areas receiving both vestibular and somatosensory signals.

Tuning of temporo-occipital activity by frontal oscillations during virtual mirror exposure causes erroneous self-recognition

Self‐face recognition, a hallmark of self‐awareness, depends on ‘off‐line’ stored information about ones face and ‘on‐line’ multisensory‐motor face‐related cues. The brain mechanisms of how on‐line sensory‐motor processes affect off‐line neural self‐face representations are unknown. This study used 3D virtual reality to create a ‘virtual mirror’ in which participants saw an avatars face moving synchronously with their own face movements. Electroencephalographic (EEG) analysis during virtual mirror exposure revealed mu oscillations in sensory‐motor cortex signalling on‐line congruency between the avatars and participants’ movements. After such exposure and compatible with a change in their off‐line self‐face representation, participants were more prone to recognize the avatars face as their own, and this was also reflected in the activation of face‐specific regions in the inferotemporal cortex. Further EEG analysis showed that the on‐line sensory‐motor effects during virtual mirror exposure caused these off‐line visual effects, revealing the brain mechanisms that maintain a coherent self‐representation, despite our continuously changing appearance.

Breathing and sense of self: Visuo-respiratory conflicts alter body self-consciousness

Bodily self-consciousness depends on the processing of interoceptive and exteroceptive signals. It can be disrupted by inducing signal conflicts. Breathing, at the crossroad between interoception and exteroception, should contribute to bodily self-consciousness. We induced visuo-respiratory conflicts in 17 subjects presented with a virtual body or a parallelepidedal object flashing synchronously or asynchronously with their breathing. A questionnaire detected illusory changes in bodily self-consciousness and breathing agency (the feeling of sensing ones breathing command). Changes in self-location were tested by measuring reaction time during mental ball drop (MBD). Synchronous illumination changed the perceived location of breathing (body: p=0.008 vs. asynchronous; object: p=0.013). It resulted in a significant change in breathing agency, but no changes in self-identification. This was corroborated by prolonged MBD reaction time (body: +0.045s, 95%CI [0.013; 0.08], p=0.007). We conclude that breathing modulates bodily self-consciousness. We also conclude that one can induce the irruption of unattended breathing into consciousness without modifying respiratory mechanics or gas exchange.

Building exposure: synergy of interaction and narration through the social channel

The ability to build and control exposure through a seamless synergy of interaction and narration is a strongly required feature of a new type of immersive-VR training and therapy system. This paper presents a practical approach for immersive-VR training and therapy applications based on interactive storytelling. It provides detailed description of a working implementation of the Interactive Narration Space (INS); this approach combines and satisfies both interaction and narration requirements through the use of high-level social interaction. By introducing the Social Channel, we aim at minimizing the contradictions between control over the story required by the trainer/therapist and interaction required by the trainee/patient. These concepts and their practical realization have been investigated in the context of emergency-situation training and psychotherapeutic exposure, and could validate the usability of mediated interaction with a virtual assistant.

Peripersonal Space: An Index of Multisensory Body–Environment Interactions in Real, Virtual, and Mixed Realities

Human-environment interactions normally occur in the physical milieu, and thus by medium of the body and within the space immediately adjacent to and surrounding the body; the peri-personal space (PPS). However, human interactions increasingly occur with or within virtual environments, and hence novel approaches and metrics must be developed to index human-environment interactions in virtual reality (VR). Here we present a multisensory task that measures the spatial extent of human PPS in real, virtual, and augmented realities. We validated it in a mixed reality ecosystem in which real environment and virtual objects are blended together in order to administer and control visual, auditory, and tactile stimuli in ecologically valid conditions. Within this mixed-reality environment, participants are asked to respond as fast as possible to tactile stimuli on their body, while task-irrelevant visual or audio-visual stimuli approach their body. Results demonstrate that, in analogy with observations derived from monkey electrophysiology and in real environmental surroundings, tactile detection is enhanced when visual or auditory stimuli are close to the body, and not when far from it. We then calculate the location where this multisensory facilitation occurs as a proxy of the boundary of PPS. We observe that mapping of PPS via audio-visual, as opposed to visual alone, looming stimuli results in sigmoidal fits – allowing for the bifurcation between near and far space – with greater goodness of fit. In sum, our approach is able to capture the boundaries of PPS on a spatial continuum, at the individual-subject level, and within a fully controlled and previously laboratory-validated setup, while maintaining the richness and ecological validity of real-life events. The task can therefore be applied to study the properties of peri-personal space in humans and to index the features governing human-environment interactions in virtual or mixed reality. We propose PPS as an ecologically valid and neurophysiologically established metric in the study of the impact of VR and related technologies on society and individuals.