Margarita Vinnikov

Gaze-contingent depth of field in realistic scenes: the user experience

Computer-generated objects presented on a display typically have the same focal distance regardless of the monocular and binocular depth cues used to portray a 3D scene. This is because they are presented on a flat screen display that has a fixed physical location. In a stereoscopic 3D display, accommodation (focus) of the eyes should always be at the distance of the screen for clear vision regardless of the depth portrayed; this fixed accommodation conflicts with vergence eye movements that the user must make to fuse stimuli located off the screen. This is known as accommodation-vergence conflict and is detrimental for user experience of stereoscopic virtual environments (VE), as it can cause visual discomfort and diplopia during use of a stereoscopic display. It is believed that, by artificially simulating focal blur and natural accommodation, it is possible to compensate for the vergence-accommodation conflict and alleviate these symptoms. We hypothesized that it is possible to compensate for conflict with a fixed accommodation cue by adding simulated focal blur according to instantaneous fixation. We examined gaze-contingent depth of field (DOF) when used in stereoscopic and non-stereoscopic 3D displays. We asked our participants to compare different conditions in terms of depth perception, image quality and viewing comfort. As expected, we found that monocular DOF gave a stronger impression of depth than no depth of field, stereoscopic cues were stronger than any kind of monocular cues, but adding depth of field to stereo displays did not enhance depth impressions. The opposite was true for image comfort. People thought that DOF impaired image quality in monocular viewing. We also observed that comfort was affected by DOF and display mode in similar fashion as image quality. However, the magnitude of the effects of DOF simulation on image quality depended on whether people associated image quality with depth or not. These results suggest that studies evaluating DOF effectiveness need to consider the type of task, type of image and questions asked.

Cue conflict between disparity change and looming in the perception of motion in depth.

We hypothesized that it is the conflict between various cues to distance that have produced results purportedly showing that vergence eye movements induced by disparity change are not an effective cue for depth. Single and compound stimuli were used to examine the perceived motion in depth (MID) produced by simulated motion oscillations specified by disparity, relative disparity, and/or looming. Estimations of the extent of MID and binocularly recorded eye movements showed that the vergence induced by disparity change is indeed an effective cue for motion in depth in conditions where looming information does not conflict with it. When looming and disparity are in conflict, looming is the stronger cue.

Real-time simulation of visual defects with gaze-contingent display

Effective management and treatment of glaucoma and other visual diseases depend on early diagnosis. However, early symptoms of glaucoma often go unnoticed until a significant portion of the visual field is lost. The ability to simulate the visual consequences of the disease offers potential benefits for patients and clinical education as well as for public awareness of its signs and symptoms. Experiments using simulated visual field defects could identify changes in behaviour, for example during driving, that one uses to compensate at the early stages of the diseases development. Furthermore, by understanding how visual field defects affect performance of visual tasks, we can help develop new strategies to cope with other devastating diseases such as macular degeneration. A Gaze-Contingent Display (GCD) system was developed to simulate an arbitrary visual field in a virtual environment. The system can estimate real-time gaze direction and eye position in earth-fixed coordinates during relatively large head movement, and thus it can be used in immersive projection based VE systems like the CAVE#8482;. Arbitrary visual fields are simulated via OpenGL and Shading Language capabilities and techniques that are supported by the GPU, thus enabling fast performance in real time. In order to simulate realistic visual defects, the system performs multiple image processing operations including change in acuity, brightness, color, glare and image distortion. The final component of the system simulates different virtual scenes that the participant can navigate through and explore. As a result, this system creates an experimental environment to study the effects of low vision on everyday tasks such as driving and navigation.

Impact of depth of field simulation on visual fatigue

While stereoscopic content can be compelling, it is not always comfortable for users to interact with on a regular basis. This is because the stereoscopic content on displays viewed at a short distance has been associated with different symptoms such as eye-strain, visual discomfort, and even nausea. Many of these symptoms have been attributed to cue conflict, for example between vergence and accommodation. To resolve those conflicts, volumetric and other displays have been proposed to improve the users experience. However, these displays are expensive, unduly restrict viewing position, or provide poor image quality. As a result, commercial solutions are not readily available. We hypothesized that some of the discomfort and fatigue symptoms exhibited from viewing in stereoscopic displays may result from a mismatch between stereopsis and blur, rather than between sensed accommodation and vergence. To find factors that may support or disprove this claim, we built a real-time gaze-contingent system that simulates depth of field (DOF) that is associated with accommodation at the virtual depth of the point of regard (POR). Subsequently, a series of experiments evaluated the impact of DOF on people of different age groups (younger versus older adults). The difference between short duration discomfort and fatigue due to prolonged viewing was also examined. Results indicated that age may be a determining factor for a users experience of DOF. There was also a major difference in a users perception of viewing comfort during short-term exposure and prolonged viewing. Primarily, people did not find that the presence of DOF enhanced short-term viewing comfort, while DOF alleviated some symptoms of visual fatigue but not all. Graphical abstractDisplay Omitted HighlightsBuilt a gaze-contingent depth-of-field (DOF) simulation for virtual reality.DOF contributes to compelling depth for non-stereoscopic displays for young adults.DOF simulation did not improve visual comfort and reduced perceived image quality.Older adults were more tolerant to DOF than young adults in terms of viewing comfort.However, adding DOF to 3D displays improved subjective measures of visual fatigue.

Psychophysics of Night Vision Device Halos

In modern Night Vision Devices (NVDs) ‘halo’ around bright light sources remains a salient imaging artifact. Although a common feature of image intensified imagery, little is known of the perceptual and operational effects of this device limitation. This paper describes two related sets of experiments. In the first set of experiments, we provide quantitative measurements of Night Vision Device (NVD) halos formed by light sources as a function of intensity and distance. This characterization allows for analysis of the possible effects of halo on human perception through NVDs. In the second set of experiments, the effects of halation on the perception of depth and environmental layout are investigated psychophysically. The custom simulation environment used and results from psychophysical experiments designed to analyze halo-induced errors in slope estimation are presented. Accurate simulation of image intensifier physics and NVD scene modeling is challenging and computationally demanding, yet needs to be performed in real-time at high frame rates and at high-resolution in advanced military simulators. Given the constraints of the real-time simulation, it is important to understand how NVD artifacts impact task performance in order to make rational engineering decisions about the required level of fidelity of the NVD simulation. A salient artifact of NVD viewing is halo, the phenomenon where the image of a bright light source appears surrounded by disc-like halo. High-fidelity physical modeling of these halo phenomena would be computationally expensive. To evaluate the level of approximation that would be sufficient for training purposes human factors data is required.

Estimating the motion-to-photon latency in head mounted displays

We present a method for estimating the Motion-to-Photon (End-to-End) latency of head mounted displays (HMDs). The specific HMD evaluated in our study was the Oculus Rift DK2, but the procedure is general. We mounted the HMD on a pendulum to introduce damped sinusoidal motion to the HMD during the pendulum swing. The latency was estimated by calculating the phase shift between the captured signals of the physical motion of the HMD and a motion-dependent gradient stimulus rendered on the display. We used the proposed method to estimate both rotational and translational Motion-to-Photon latencies of the Oculus Rift DK2.

Effects of image intensifier halo on perceived layout

Night vision devices (NVDs) or night-vision goggles (NVGs) based on image intensifiers improve nighttime visibility and extend night operations for military and increasingly civil aviation. However, NVG imagery is not equivalent to daytime vision and impaired depth and motion perception has been noted. One potential cause of impaired perceptions of space and environmental layout is NVG halo, where bright light sources appear to be surrounded by a disc-like halo. In this study we measured the characteristics of NVG halo psychophysically and objectively and then evaluated the influence of halo on perceived environmental layout in a simulation experiment. Halos are generated in the device and are not directly related to the spatial layout of the scene. We found that, when visible, halo image (i.e. angular) size was only weakly dependent on both source intensity and distance although halo intensity did vary with effective source intensity. The size of halo images surrounding lights sources are independent of the source distance and thus do not obey the normal laws of perspective. In simulation experiments we investigated the effect of NVG halo on judgements of observer attitude with respect to the ground during simulated flight. We discuss the results in terms of NVG design and of the ability of human operators to compensate for perceptual distortions.

Gaze-Contingent Auditory Displays for Improved Spatial Attention in Virtual Reality

Virtual reality simulations of group social interactions are important for many applications, including the virtual treatment of social phobias, crowd and group simulation, collaborative virtual environments (VEs), and entertainment. In such scenarios, when compared to the real world, audio cues are often impoverished. As a result, users cannot rely on subtle spatial audio-visual cues that guide attention and enable effective social interactions in real-world situations. We explored whether gaze-contingent audio enhancement techniques driven by inferring audio-visual attention in virtual displays could be used to enable effective communication in cluttered audio VEs. In all of our experiments, we hypothesized that visual attention could be used as a tool to modulate the quality and intensity of sounds from multiple sources to efficiently and naturally select spatial sound sources. For this purpose, we built a gaze-contingent display (GCD) that allowed tracking of a user’s gaze in real-time and modifying the volume of the speakers’ voices contingent on the current region of overt attention. We compared six different techniques for sound modulation with a base condition providing no attentional modulation of sound. The techniques were compared in terms of source recognition and preference in a set of user studies. Overall, we observed that users liked the ability to control the sounds with their eyes. They felt that a rapid change in attenuation with attention but not the elimination of competing sounds (partial rather than absolute selection) was most natural. In conclusion, audio GCDs offer potential for simulating rich, natural social, and other interactions in VEs. They should be considered for improving both performance and fidelity in applications related to social behaviour scenarios or when the user needs to work with multiple audio sources of information.

Heading Perception with Simulated Visual Defects

Heading perception depends on the ability of different regions of the visual field to extract accurate information about the direction of the visual flow. Hence due to its ability to extract the most accurate information, the central visual field plays a major role in heading estimation. With experience people learn to utilize other regions especially if there is central field loss/impairment. Nevertheless, it is not clear what happens when information in central vision becomes altered or cannot be picked up. In the present study, we examined the effects of gaze-contingent alteration of regions of the visual field on heading. On each trial, one of six different directions of self-motion were simulated (headings ±7.5°, ±5.0° and ±2.5° from the centre of the screen). The simulated defects were analogous to two typical visual field disturbances resulting from macular degeneration, either metamorphopsia or scotomas. Specifically, with a force choice procedure we compared performance with no visual defects to that with five different simulated defects (either 5° or 10° horizontal perturbations, 5° or 10° Gaussian perturbations, or a 10° scotoma). We also looked at three gaze conditions - free viewing, directional viewing and tracking features in the scene. Heading performance was not significantly different in the two environments examined (translation over a plane covered with blue particles or through a forest). Performance declined in the presence of simulated visual defects, as well as when they were instructed to visually track specific scene features. Performance was most accurate for all heading directions during the free view conditions. We conclude that when people are free to direct their gaze in the scene they are able to minimize the impact of simulated central visual field loss/distortion. Meeting abstract presented at VSS 2015.

Contingency evaluation of gaze-contingent displays for real-time visual field simulations

The visual field is the area of space that can be seen when an observer fixates a given point. Many visual capabilities vary with position in the visual field and many diseases result in changes in the visual field. With current technology, it is possible to build very complex real-time visual field simulations that employ gaze-contingent displays. Nevertheless, there are still no established techniques to evaluate such systems. We have developed a method to evaluate a systems contingency by employing visual blind spot localization as well as foveal fixation. During the experiment, gaze-contingent and static conditions were compared. There was a strong correlation between predicted results and gaze-contingent trials. This evaluation method can also be used with patient populations and for the evaluation of gaze-contingent display systems, when there is need to evaluate a visual field outside of the foveal region.