David J. Zielinski

Evaluating Display Fidelity and Interaction Fidelity in a Virtual Reality Game

In recent years, consumers have witnessed a technological revolution that has delivered more-realistic experiences in their own homes through high-definition, stereoscopic televisions and natural, gesture-based video game consoles. Although these experiences are more realistic, offering higher levels of fidelity, it is not clear how the increased display and interaction aspects of fidelity impact the user experience. Since immersive virtual reality (VR) allows us to achieve very high levels of fidelity, we designed and conducted a study that used a six-sided CAVE to evaluate display fidelity and interaction fidelity independently, at extremely high and low levels, for a VR first-person shooter (FPS) game. Our goal was to gain a better understanding of the effects of fidelity on the user in a complex, performance-intensive context. The results of our study indicate that both display and interaction fidelity significantly affect strategy and performance, as well as subjective judgments of presence, engagement, and usability. In particular, performance results were strongly in favor of two conditions: low-display, low-interaction fidelity (representative of traditional FPS games) and high-display, high-interaction fidelity (similar to the real world).

Extinction in multiple virtual reality contexts diminishes fear reinstatement in humans

Although conditioned fear can be effectively extinguished by unreinforced exposure to a threat cue, fear responses tend to return when the cue is encountered some time after extinction (spontaneous recovery), in a novel environment (renewal), or following presentation of an aversive stimulus (reinstatement). As extinction represents a context-dependent form of new learning, one possible strategy to circumvent the return of fear is to conduct extinction across several environments. Here, we tested the effectiveness of multiple context extinction in a two-day fear conditioning experiment using 3-D virtual reality technology to create immersive, ecologically-valid context changes. Fear-potentiated startle served as the dependent measure. All three experimental groups initially acquired fear in a single context. A multiple extinction group then underwent extinction in three contexts, while a second group underwent extinction in the acquisition context and a third group underwent extinction in a single different context. All groups returned 24h later to test for return of fear in the extinction context (spontaneous recovery) and a novel context (renewal and reinstatement/test). Extinction in multiple contexts attenuated reinstatement of fear but did not reduce spontaneous recovery. Results from fear renewal were tendential. Our findings suggest that multi-context extinction can reduce fear relapse following an aversive event--an event that often induces return of fear in real-world settings--and provides empirical support for conducting exposure-based clinical treatments across a variety of environments.

Effects of virtual environment platforms on emotional responses

The goal of the current study was to investigate the effects of different virtual environment (VE) technologies (i.e., desktop, head mounted display, or fully immersive platforms) on emotional arousal and task performance. Fifty-three participants were recruited from a college population. Reactivity to stressful VEs was examined in three VE systems from desktop to high-end fully immersive systems. The experiment was a 3 (desktop system, head mounted display, and six wall system)×2 (high- and low-stressful VE) within subject design, with self-reported emotional arousal and valence, skin conductance, task performance, presence, and simulator sickness examined as dependent variables. Replicating previous studies, the fully immersive system induced the highest sense of presence and the head mounted display system elicited the highest amount of simulator sickness. Extending previous studies, the results demonstrated that VE platforms were associated with different patterns in emotional responses and task performance. Our findings suggest that different VE systems may be appropriate for different scientific purposes when studying stress reactivity using emotionally evocative tasks.

Revealing Context-Specific Conditioned Fear Memories with Full Immersion Virtual Reality

The extinction of conditioned fear is known to be context-specific and is often considered more contextually bound than the fear memory itself (Bouton, 2004). Yet, recent findings in rodents have challenged the notion that contextual fear retention is initially generalized. The context-specificity of a cued fear memory to the learning context has not been addressed in the human literature largely due to limitations in methodology. Here we adapt a novel technology to test the context-specificity of cued fear conditioning using full immersion 3-D virtual reality (VR). During acquisition training, healthy participants navigated through virtual environments containing dynamic snake and spider conditioned stimuli (CSs), one of which was paired with electrical wrist stimulation. During a 24-h delayed retention test, one group returned to the same context as acquisition training whereas another group experienced the CSs in a novel context. Unconditioned stimulus expectancy ratings were assayed on-line during fear acquisition as an index of contingency awareness. Skin conductance responses time-locked to CS onset were the dependent measure of cued fear, and skin conductance levels during the interstimulus interval were an index of context fear. Findings indicate that early in acquisition training, participants express contingency awareness as well as differential contextual fear, whereas differential cued fear emerged later in acquisition. During the retention test, differential cued fear retention was enhanced in the group who returned to the same context as acquisition training relative to the context shift group. The results extend recent rodent work to illustrate differences in cued and context fear acquisition and the contextual specificity of recent fear memories. Findings support the use of full immersion VR as a novel tool in cognitive neuroscience to bridge rodent models of contextual phenomena underlying human clinical disorders.

Medial prefrontal pathways for the contextual regulation of extinguished fear in humans

The maintenance of anxiety disorders is thought to depend, in part, on deficits in extinction memory, possibly due to reduced contextual control of extinction that leads to fear renewal. Animal studies suggest that the neural circuitry responsible fear renewal includes the hippocampus, amygdala, and dorsomedial (dmPFC) and ventromedial (vmPFC) prefrontal cortex. However, the neural mechanisms of context-dependent fear renewal in humans remain poorly understood. We used functional magnetic resonance imaging (fMRI), combined with psychophysiology and immersive virtual reality, to elucidate how the hippocampus, amygdala, and dmPFC and vmPFC interact to drive the context-dependent renewal of extinguished fear. Healthy human participants encountered dynamic fear-relevant conditioned stimuli (CSs) while navigating through 3-D virtual reality environments in the MRI scanner. Conditioning and extinction were performed in two different virtual contexts. Twenty-four hours later, participants were exposed to the CSs without reinforcement while navigating through both contexts in the MRI scanner. Participants showed enhanced skin conductance responses (SCRs) to the previously-reinforced CS+ in the acquisition context on Day 2, consistent with fear renewal, and sustained responses in the dmPFC. In contrast, participants showed low SCRs to the CSs in the extinction context on Day 2, consistent with extinction recall, and enhanced vmPFC activation to the non-reinforced CS-. Structural equation modeling revealed that the dmPFC fully mediated the effect of the hippocampus on right amygdala activity during fear renewal, whereas the vmPFC partially mediated the effect of the hippocampus on right amygdala activity during extinction recall. These results indicate dissociable contextual influences of the hippocampus on prefrontal pathways, which, in turn, determine the level of reactivation of fear associations.

Exploring Semantic Social Networks Using Virtual Reality

We present Redgraph, the first generic virtual reality visualization program for Semantic Web data. Redgraph is capable of handling large data-sets, as we demonstrate on social network data from the U.S. Patent Trade Office. We develop a Semantic Web vocabulary of virtual reality terms compatible with GraphXML to map graph visualization into the Semantic Web itself. Our approach to visualizing Semantic Web data takes advantage of user-interaction in an immersive environment to bypass a number of difficult issues in 3-dimensional graph visualization layout by relying on users themselves to interactively extrude the nodes and links of a 2-dimensional graph into the third dimension. When users touch nodes in the virtual reality environment, they retrieve data formatted according to the datas schema or ontology. We applied Redgraph to social network data constructed from patents, inventors, and institutions from the United States Patent and Trademark Office in order to explore networks of innovation in computing. Using this data-set, results of a user study comparing extrusion (3-D) vs. no-extrusion (2-D) are presented. The study showed the use of a 3-D interface by subjects led to significant improvement on answering of fine-grained questions about the data-set, but no significant difference was found for broad questions about the overall structure of the data. Furthermore, inference can be used to improve the visualization, as demonstrated with a data-set of biotechnology patents and researchers.

Spatial proximity amplifies valence in emotional memory and defensive approach-avoidance

In urban areas, people often have to stand or move in close proximity to others. The egocentric distance to stimuli is a powerful determinant of defensive behavior in animals. Yet, little is known about how spatial proximity to others alters defensive responses in humans. We hypothesized that the valence of social cues scales with egocentric distance, such that proximal social stimuli have more positive or negative valence than distal stimuli. This would predict enhanced defensive responses to proximal threat and reduced defensive responses to proximal reward. We tested this hypothesis across four experiments using 3-D virtual reality simulations. Results from Experiment 1 confirmed that proximal social stimuli facilitate defensive responses, as indexed by fear-potentiated startle, relative to distal stimuli. Experiment 2 revealed that interpersonal defensive boundaries flexibly increase with aversive learning. Experiment 3 examined whether spatial proximity enhances memory for aversive experiences. Fear memories for social threats encroaching on the body were more persistent than those acquired at greater interpersonal distances, as indexed by startle. Lastly, Experiment 4 examined how egocentric distance influenced startle responses to social threats during defensive approach and avoidance. Whereas fear-potentiated startle increased with proximity when participants actively avoided receiving shocks, startle decreased with proximity when participants tolerated shocks to receive monetary rewards, implicating opposing gradients of distance on threat versus reward. Thus, proximity in egocentric space amplifies the valence of social stimuli that, in turn, facilitates emotional memory and approach-avoidance responses. These findings have implications for understanding the consequences of increased urbanization on affective interpersonal behavior.

Shadow walking: An unencumbered locomotion technique for systems with under-floor projection

When viewed from below, a users feet cast shadows onto the floor screen of an under-floor projection system, such as a six-sided CAVE. Tracking those shadows with a camera provides enough information for calculating a users ground-plane location, foot orientation, and footstep events. We present Shadow Walking, an unencumbered locomotion technique that uses shadow tracking to sense a users walking direction and step speed. Shadow Walking affords virtual locomotion by detecting if a user is walking in place. In addition, Shadow Walking supports a sidestep gesture, similar to the iPhones pinch gesture. In this paper, we describe how we implemented Shadow Walking and present a preliminary assessment of our new locomotion technique. We have found Shadow Walking provides advantages of being unencumbered, inexpensive, and easy to implement compared to other walking-in-place approaches. It also has potential for extended gestures and multi-user locomotion.

KinImmerse: Macromolecular VR for NMR ensembles

BackgroundIn molecular applications, virtual reality (VR) and immersive virtual environments have generally been used and valued for the visual and interactive experience – to enhance intuition and communicate excitement – rather than as part of the actual research process. In contrast, this work develops a software infrastructure for research use and illustrates such use on a specific case.MethodsThe Syzygy open-source toolkit for VR software was used to write the KinImmerse program, which translates the molecular capabilities of the kinemage graphics format into software for display and manipulation in the DiVE (Duke immersive Virtual Environment) or other VR system. KinImmerse is supported by the flexible display construction and editing features in the KiNG kinemage viewer and it implements new forms of user interaction in the DiVE.ResultsIn addition to molecular visualizations and navigation, KinImmerse provides a set of research tools for manipulation, identification, co-centering of multiple models, free-form 3D annotation, and output of results. The molecular research test case analyzes the local neighborhood around an individual atom within an ensemble of nuclear magnetic resonance (NMR) models, enabling immersive visual comparison of the local conformation with the local NMR experimental data, including target curves for residual dipolar couplings (RDCs).ConclusionThe promise of KinImmerse for production-level molecular research in the DiVE is shown by the locally co-centered RDC visualization developed there, which gave new insights now being pursued in wider data analysis.

Comparison of interactive environments for the archaeological exploration of 3D landscape data

The increasingly widespread availability of high-accuracy terrain models is revolutionizing our understanding of historic landscapes across the globe, yet much of this inherently 3D data is viewed and analyzed using 2D Geographical Information System (GIS). The ability to explore the environments in a more immersive way that takes advantage of the full data content is advantageous for professionals and researchers, but is also highly desirable for education and public outreach. This paper describes the method and outcomes of a comparison of three virtual environments; a six-sided CAVE-type immersive virtual reality system (referred to henceforth as CAVE); a 3D web application and a standard 2D desktop paradigm in the form of a GIS. Two groups of participants were used to reflect specialist and non-specialist interests. This study showed that while the 2D GIS, the most common interface for exploring archaeological data, is well-suited to expert interpretation (based on previous familiarity with the system), it is significantly harder for non-specialists to undertake a feature identification and location task in this environment when compared with the 3D environments. Specialist users also mostly preferred the ability to view terrain data in 3D. The experience of fully-immersive CAVE-type system was valuable for a sense of place and contextualizing features in a way that was not possible in the other environments. However it was not shown that this led to improved archaeological observations during the exploration and there is some evidence that the lack of orientation made recounting features in the reflection time more difficult. Although small-scale the experiment gave valuable insight into the use of the different environments by specialist and non-specialist groups, allowing the 3D web application to be identified as the optimal environment for pedagogical purposes.