J Schulte-Pelkum

Moving sounds enhance the visually-induced self-motion illusion (circular vection) in virtual reality

While rotating visual and auditory stimuli have long been known to elicit self-motion illusions (“circular vection”), audiovisual interactions have hardly been investigated. Here, two experiments investigated whether visually induced circular vection can be enhanced by concurrently rotating auditory cues that match visual landmarks (e.g., a fountain sound). Participants sat behind a curved projection screen displaying rotating panoramic renderings of a market place. Apart from a no-sound condition, headphone-based auditory stimuli consisted of mono sound, ambient sound, or low-/high-spatial resolution auralizations using generic head-related transfer functions (HRTFs). While merely adding nonrotating (mono or ambient) sound showed no effects, moving sound stimuli facilitated both vection and presence in the virtual environment. This spatialization benefit was maximal for a medium (20° × 15°) FOV, reduced for a larger (54° × 45°) FOV and unexpectedly absent for the smallest (10° × 7.5°) FOV. Increasing auralization spatial fidelity (from low, comparable to five-channel home theatre systems, to high, 5° resolution) provided no further benefit, suggesting a ceiling effect. In conclusion, both self-motion perception and presence can benefit from adding moving auditory stimuli. This has important implications both for multimodal cue integration theories and the applied challenge of building affordable yet effective motion simulators.

Cognitive factors can influence self-motion perception (vection) in virtual reality

Research on self-motion perception and simulation has traditionally focused on the contribution of physical stimulus properties (“bottom-up factors”) using abstract stimuli. Here, we demonstrate that cognitive (“top-down”) mechanisms like ecological relevance and presence evoked by a virtual environment can also enhance visually induced self-motion illusions (vection). In two experiments, naive observers were asked to rate presence and the onset, intensity, and convincingness of circular vection induced by different rotating visual stimuli presented on a curved projection screen (FOV: 54° × 45°). Globally consistent stimuli depicting a natural 3D scene proved more effective in inducing vection and presence than inconsistent (scrambled) or unnatural (upside-down) stimuli with similar physical stimulus properties. Correlation analyses suggest a direct relationship between spatial presence and vection. We propose that the coherent pictorial depth cues and the spatial reference frame evoked by the naturalistic environment increased the believability of the visual stimulus, such that it was more easily accepted as a stable “scene” with respect to which visual motion is more likely to be judged as self-motion than object motion. This work extends our understanding of mechanisms underlying self-motion perception and might thus help to improve the effectiveness and believability of virtual reality applications.

Towards lean and elegant self-motion simulation in virtual reality

Despite recent technological advances, convincing self-motion simulation in virtual reality (VR) is difficult to achieve, and users often suffer from motion sickness and/or disorientation in the simulated world. Instead of trying to simulate self-motions with physical realism (as is often done for, e.g., driving or flight simulators), we propose in this paper a perceptually oriented approach towards self-motion simulation. Following this paradigm, we performed a series of psychophysical experiments to determine essential visual, auditory, and vestibular/tactile parameters for an effective and perceptually convincing self-motion simulation. These studies are a first step towards our overall goal of achieving lean and elegant self-motion simulation in virtual reality (VR) without physically moving the observer. In a series of psychophysical experiments about the self-motion illusion (circular vection), we found that (i) vection as well as presence in the simulated environment is increased by a consistent, naturalistic visual scene when compared to a sliced, inconsistent version of the identical scene, (ii) barely noticeable marks on the projection screen can increase vection as well as presence in an unobtrusive manner, (iii) physical vibrations of the observers seat can enhance the vection illusion, and (iv) spatialized 3D audio cues embedded in the simulated environment increase the sensation of self-motion and presence. We conclude that providing consistent cues about self-motion to multiple sensory modalities can enhance vection, even if physical motion cues are absent. These results yield important implications for the design of lean and elegant self-motion simulators.

Simulating believable forward accelerations on a stewart motion platform

It is still an unsolved problem how to optimally simulate self-motion using motion simulators. We investigated how a forward acceleration can be simulated as believably as possible on a hexapod motion platform equipped with a projection screen. Human participants rated the believability of brief forward accelerations. These were simulated as visual forward accelerations over a ground plane with people as size cues, presented together with brief forward surge translations and backward pitches of the platform, and synchronous random up--down movements of the camera in the visual scene and the platform. The magnitudes of all of the parameters were varied independently across trials. Even though variability between participants was high, the most believable simulation occurred when visual accelerations were combined with backward pitches of the platform, which changed the gravitoinertial vector direction approximately consistent with the visual acceleration. However, a wide range of platform pitches was accepted as believable. With high visual acceleration cues most participants reported trials as realistic even when the platform tilt rate was above vestibular canal thresholds reported in other works. Other manipulated parameters had only a mild influence on the responses. These results can be used to optimize motion-cueing algorithms for simulating linear accelerations in motion simulators.

Technical Section: Circular, linear, and curvilinear vection in a large-screen virtual environment with floor projection

Vection is defined as the compelling sensation of illusory self- motion elicited by a moving sensory, usually visual, stimulus. This paper presents collected introspective data on the experience of linear, circular, and curvilinear vection. We evaluate the differences between twelve different trajectories and the influence of the floor projection on the illusion of self-motion. All of the simulated self- motions examined are of a constant velocity, except for a brief simulated initial acceleration. First, we find that linear translations to the left and right are perceived as the least convincing, while linear down is perceived as the most convincing of the linear trajectories. Second, we find that the floor projection significantly improves the introspective measures of linear vection experienced in a photorealistic three-dimensional town. Finally, we find that while linear forward vection is not perceived to be very convincing, curvilinear forward vection is reported to be as convincing as circular vection. Considering our experimental results, our suggestions for simulators and VE applications where vection is desirable is to increase the number of curvilinear trajectories (as opposed to linear ones) and, if possible, add floor projection in order to improve the illusory sense of self-motion.

Perceptual and Cognitive Factors for Self-Motion Simulation in Virtual Environments: How Can Self-Motion Illusions (“Vection”) Be Utilized?

How can we convincingly simulate observer locomotion through virtual environments without having to allow for full physical observer movement? That is, how can we best utilize multi-modal stimulation to provide the compelling illusion of moving through simulated worlds while reducing the overall simulation effort? This chapter provides a review on the contribution and interaction of visual, auditory, vibrational, and biomechanical cues (e.g., walking) for self-motion perception and simulation in VR. We propose an integrative framework and discuss potential synergistic effects of perceptual and cognitive influences on self-motion perception in VEs. Based on this perspective, we envision a lean-and-elegant approach that utilizes multi-modal self-motion illusions and perceptual-cognitive factors in a synergistic manner to improve perceptual and behavioral effectiveness and reduce the demand for physical (loco-)motion interfaces to a more affordable level.

The effect of cognition on the visually-induced illusion of self-motion (vection)

Stimuli depicting a natural scene can produce faster, stronger, and more convincing sensation of illusory selfmotion The results of the experiment revealed that a stimulus depicting a natural scene can produce faster, stronger, and more convincing sensation of illusory self-motion. Previous studies have typically used abstract stimuli to induce vection. Here, we show that the illusion can be enhanced if a natural scene is used instead. A possible explanation for why this happens is that natural scenes are less likely to be interpreted as moving because of the assumption of a stable environment (Dichgans & Brandt, 1978).

An Integrative Approach to Presence and Self-Motion Perception Research

This chapter is concerned with the perception and simulation of self-motion in virtual environments, and how spatial presence and other higher cognitive and top-down factors can contribute to improve the illusion of self-motion (“vection”) in virtual reality (VR). In the real world, we are used to being able to move around freely and interact with our environment in a natural and effortless manner. Current VR technology does, however, hardly allow for natural, life-like interaction between the user and the virtual environment. One crucial shortcoming is the insufficient and often unconvincing simulation of self-motion, which frequently causes disorientation, unease, and motion sickness. The specific focus of this chapter is the investigation of potential relations between higher-level factors like presence on the one hand and self-motion perception in VR on the other hand. Even though both presence and self-motion illusions have been extensively studied in the past, the question whether/how they might be linked to one another has received relatively little attention by researchers so far. After reviewing relevant literature on vection and presence, we present data from two experiments, which explicitly investigated potential relations between vection and presence and indicate that there might indeed be a direct link between these two phenomena. We discuss theoretical and practical implications from these findings and conclude by sketching a tentative theoretical framework that discusses how a broadened view that incorporates both presence and vection research might lead to a better understanding of both phenomena, and might ultimately be employed to improve not only the perceptual effectiveness of a given VR simulation, but also its behavioural and goal/application-specific effectiveness.

Perceiving Simulated Ego-Motions in Virtual Reality - Comparing Large Screen Displays with HMDs

In Virtual Reality, considerable systematic spatial orientation problems frequently occur that do not happen in comparable real-world situations. This study investigated possible origins of these problems by examining the influence of visual field of view (FOV) and type of display device (head-mounted display (HMD) vs. projection screens) on basic human spatial orientation behavior. In Experiment 1, participants had to reproduce traveled distances and to turn specified target angles in a simple virtual environment without any landmarks that was projected onto a 180° half-cylindrical projection screen. As expected, distance reproduction performance showed only small systematic errors. Turning performance, however, was unexpectedly almost perfect (gain=0.97), with negligible systematic errors and minimal variability, which is unprecedented in the literature. In Experiment 2, turning performance was compared between a projection screen (FOV 84°×63°), an HMD (40°×30°), and blinders (40°×30°) that restricted the FOV on the screen. Performance was best with the screen (gain 0.77) and worst with the HMD (gain 0.57). We found a significant difference between blinders (gain 0.73) and HMD, which indicates that different display devices can influence ego-motion perception differentially, even if the physical FOVs are equal. We conclude that the type of display device (HMD vs. curved projection screen) seems to be more critical than the FOV for the perception of ego-rotations. Furthermore, large, curved projection screens yielded better performance than HMDs.

Circular, Linear, and Curvilinear Vection in a Large-screen Virtual Environment with Floor Projection

Vection is defined as the compelling sensation of illusory self- motion elicited by a moving sensory, usually visual, stimulus. This paper presents collected introspective data on the experience of linear, circular, and curvilinear vection. We evaluate the differences between twelve different trajectories and the influence of the floor projection on the illusion of self-motion. All of the simulated self- motions examined are of a constant velocity, except for a brief simulated initial acceleration. First, we find that linear translations to the left and right are perceived as the least convincing, while linear down is perceived as the most convincing of the linear trajectories. Second, we find that the floor projection significantly improves the introspective measures of linear vection experienced in a photorealistic three-dimensional town. Finally, we find that while linear forward vection is not perceived to be very convincing, curvilinear forward vection is reported to be as convincing as circular vection. Considering our experimental results, our suggestions for simulators and VE applications where vection is desirable is to increase the number of curvilinear trajectories (as opposed to linear ones) and, if possible, add floor projection in order to improve the illusory sense of self-motion.