Simon J. Watt

Slant from texture and disparity cues: optimal cue combination.

How does the visual system combine information from different depth cues to estimate three-dimensional scene parameters? We tested a maximum-likelihood estimation (MLE) model of cue combination for perspective (texture) and binocular disparity cues to surface slant. By factoring the reliability of each cue into the combination process, MLE provides more reliable estimates of slant than would be available from either cue alone. We measured the reliability of each cue in isolation across a range of slants and distances using a slant-discrimination task. The reliability of the texture cue increases as |slant| increases and does not change with distance. The reliability of the disparity cue decreases as distance increases and varies with slant in a way that also depends on viewing distance. The trends in the single-cue data can be understood in terms of the information available in the retinal images and issues related to solving the binocular correspondence problem. To test the MLE model, we measured perceived slant of two-cue stimuli when disparity and texture were in conflict and the reliability of slant estimation when both cues were available. Results from the two-cue study indicate, consistent with the MLE model, that observers weight each cue according to its relative reliability: Disparity weight decreased as distance and |slant| increased. We also observed the expected improvement in slant estimation when both cues were available. With few discrepancies, our data indicate that observers combine cues in a statistically optimal fashion and thereby reduce the variance of slant estimates below that which could be achieved from either cue alone. These results are consistent with other studies that quantitatively examined the MLE model of cue combination. Thus, there is a growing empirical consensus that MLE provides a good quantitative account of cue combination and that sensory information is used in a manner that maximizes the precision of perceptual estimates.

Focus Cues Affect Perceived Depth

Depth information from focus cues--accommodation and the gradient of retinal blur--is typically incorrect in three-dimensional (3-D) displays because the light comes from a planar display surface. If the visual system incorporates information from focus cues into its calculation of 3-D scene parameters, this could cause distortions in perceived depth even when the 2-D retinal images are geometrically correct. In Experiment 1 we measured the direct contribution of focus cues to perceived slant by varying independently the physical slant of the display surface and the slant of a simulated surface specified by binocular disparity (binocular viewing) or perspective/texture (monocular viewing). In the binocular condition, slant estimates were unaffected by display slant. In the monocular condition, display slant had a systematic effect on slant estimates. Estimates were consistent with a weighted average of slant from focus cues and slant from disparity/texture, where the cue weights are determined by the reliability of each cue. In Experiment 2, we examined whether focus cues also have an indirect effect on perceived slant via the distance estimate used in disparity scaling. We varied independently the simulated distance and the focal distance to a disparity-defined 3-D stimulus. Perceived slant was systematically affected by changes in focal distance. Accordingly, depth constancy (with respect to simulated distance) was significantly reduced when focal distance was held constant compared to when it varied appropriately with the simulated distance to the stimulus. The results of both experiments show that focus cues can contribute to estimates of 3-D scene parameters. Inappropriate focus cues in typical 3-D displays may therefore contribute to distortions in perceived space.

A stereo display prototype with multiple focal distances

Typical stereo displays provide incorrect focus cues because the light comes from a single surface. We describe a prototype stereo display comprising two independent fixed-viewpoint volumetric displays. Like autostereoscopic volumetric displays, fixed-viewpoint volumetric displays generate near-correct focus cues without tracking eye position, because light comes from sources at the correct focal distances. (In our prototype, from three image planes at different physical distances.) Unlike autostereoscopic volumetric displays, however, fixed-viewpoint volumetric displays retain the qualities of modern projective graphics: view-dependent lighting effects such as occlusion, specularity, and reflection are correctly depicted; modern graphics processor and 2-D display technology can be utilized; and realistic fields of view and depths of field can be implemented. While not a practical solution for general-purpose viewing, our prototype display is a proof of concept and a platform for ongoing vision research. The design, implementation, and verification of this stereo display are described, including a novel technique of filtering along visual lines using 1-D texture mapping.

Technical Section: A review of virtual environments for training in ball sports

There is growing interest in utilising virtual environments (VEs) in the context of sports. In particular there is a desire to be able to improve sensorimotor skills rather than just using a VE as a tool for strategy analysis, or entertainment. The range of skills required across all different sports is very large and varied. This review of the state-of-the-art, therefore focuses on just ball sports, and was carried out as part of a current project developing training tools for rugby. A VE needs to provide realistic rendering of the sports scene to achieve good perceptual fidelity. More important for a sport-themed VE is high functional fidelity, which requires an accurate physics model of a complex environment, real time response, and a natural user interface. The goal is to provide multiple scenarios to players at different levels of difficulty, providing them with improved skills that can be applied directly to the real sports arena. The typical hardware and software components needed are identified in the paper, and important psychological factors that should be considered are discussed. The challenges that need to be overcome are identified and illustrated with examples from recent projects in this area.

The visual control of reaching and grasping: binocular disparity and motion parallax.

The primary visual sources of depth and size information are binocular cues and motion parallax. Here, the authors determine the efficacy of these cues to control prehension by presenting them in isolation from other visual cues. When only binocular cues were available, reaches showed normal scaling of the transport and grasp components with object distance and size. However, when only motion parallax was available, only the transport component scaled reliably. No additional increase in scaling was found when both cues were available simultaneously. Therefore, although equivalent information is available from binocular and motion parallax information, the latter may be of relatively limited use for the control of the grasp. Binocular disparity appears selectively important for the control of the grasp.

A dissociation of perception and action in normal human observers: the effect of temporal-delay.

Neuropsychological results support the proposal that the human visual system is organised into distinct processing pathways, one for conscious perception and one for the control of action. Here, we compare perceptual and action responses following a pre-response-delay. Experiment 1 required participants to reproduce remembered locations and found that although perceptual matches were unaffected by delays of up to 4 s, pointing responses were significantly biased after only 2 s. Experiment 2 examined whether both the transport and grasp components of a natural prehensile movement were similarly affected by delay. Both peak wrist velocities and peak grip-apertures were affected equivalently by delay, suggesting that the two components of a prehensile movement have similar temporal constraints. The results from both experiments are consistent with the general perception-action dichotomy as originally proposed by Milner and Goodale [The visual brain in action, Oxford: Oxford University Press, 1995].

Field of view affects reaching, not grasping

Abstract. It has been observed that wearing goggles that restrict the field of view (FOV) causes familiar objects to appear both smaller and nearer. To investigate this further, we examined the effect of a range of field sizes (4°, 8°, 16°, 32° and 64°) on estimates of object distance and object size used to control reaching and grasping movements of binocular observers. No visual or haptic feedback was available during the experiment. It was found that, as the FOV was decreased, the distance reached by subjects also decreased, whereas the size of their grasp was unaffected. In a second experiment, we compared reaching and grasping responses under binocular and monocular conditions for 8° and 64° field sizes and show that the effects of FOV do not result from the progressive loss of binocular information. We conclude that reducing the FOV produces substantial and dissociable effects on reaching and grasping behaviour and that field size must be taken into account in any context where visuo-motor performance is important.

The role of binocular vision in grasping: a small stimulus-set distorts results

The role of binocular vision in grasping has frequently been assessed by measuring the effects on grasp kinematics of covering one eye. These studies have typically used three or fewer objects presented at three or fewer distances, raising the possibility that participants learn the properties of the stimulus set. If so, even relatively poor visual information may be sufficient to identify which object/distance configuration is presented on a given trial, in effect providing an additional source of depth information. Here we show that the availability of this uncontrolled cue leads to an underestimate of the effects of removing binocular information, and therefore to an overestimate of the effectiveness of the remaining cues. We measured the effects of removing binocular cues on visually open-loop grasps using (1) a conventional small stimulus-set, and (2) a large, pseudo-randomised stimulus set, which could not be learned. Removing binocular cues resulted in a significant change in grip aperture scaling in both conditions: peak grip apertures were larger (when reaching to small objects), and scaled less with increases in object size. However, this effect was significantly larger with the randomised stimulus set. These results confirm that binocular information makes a significant contribution to grasp planning. Moreover, they suggest that learned stimulus information can contribute to grasping in typical experiments, and so the contribution of information from binocular vision (and from other depth cues) may not have been measured accurately.

Binocular vision and prehension in middle childhood.

Binocular cues have been shown previously to make an important contribution to the control of natural prehensile movements in adults [Visual Cognition 4 (1997) 113, Vision Research 32 (1992) 1513, Neuropsychologia 38 (2000) 1473]. The present study examined the role of binocular vision in the control of prehension in middle childhood. Fourteen children aged 5-6 years, and 16 children aged 10-11 years reached out and grasped different sized objects at different distances, in either binocular or monocular viewing conditions. In contrast to adult data, many of the principal kinematic indices of the childrens reaches were unaffected by the removal of binocular information. The older children, like adults, spent an increased amount of time in the final approach to the object when only monocular information was available. However, both peak wrist velocities and peak grip apertures were unaffected by the removal of binocular information and continued to scale with object properties in the normal way. These results suggest that the use of binocular cues to control prehensile movements is not yet mature at the age of 10-11 years.

Achieving near-correct focus cues in a 3D display using multiple image planes

Focus cues specify inappropriate 3-D scene parameters in conventional displays because the light comes from a single surface, independent of the depth relations in the portrayed scene. This can lead to distortions in perceived depth, as well as discomfort and fatigue due to the differing demands on accommodation and vergence. Here we examine the efficacy of a stereo-display prototype designed to minimize these problems by using multiple image planes to present near-correct focus cues. Each eye’s view is the sum of several images presented at different focal distances. Image intensities are assigned based on the dioptric distance of each image plane from the portrayed object, determined along visual lines. The stimulus to accommodation is more consistent with the portrayed depth than with conventional displays, but it still differs from the stimulus in equivalent real scenes. Compared to a normal, fixed-distance display, observers showed improved stereoscopic performance in different psychophysical tasks including speed of fusing stereoscopic images, precision of depth discrimination, and accuracy of perceived depth estimates. The multiple image-planes approach provides a practical solution for some shortcomings of conventional displays.