James T. Todd

The effects of viewing angle, camera angle, and sign of surface curvature on the perception of three-dimensional shape from texture

Computational models for determining three-dimensional shape from texture based on local foreshortening or gradients of scaling are able to achieve accurate estimates of surface relief from an image when it is observed from the same visual angle with which it was photographed or rendered. These models produce conflicting predictions, however, when an image is viewed from a different visual angle. An experiment was performed to test these predictions, in which observers judged the apparent depth profiles of hyperbolic cylinders under a wide variety of conditions. The results reveal that the apparent patterns of relief from texture are systematically underestimated; convex surfaces appear to have greater depth than concave surfaces, large camera angles produce greater amounts of perceived depth than small camera angles, and the apparent depth-to-width ratio for a given image of a surface is greater for small viewing angles than for large viewing angles. Because these results are incompatible with all existing computational models, a new model is presented based on scaling contrast that can successfully account for all aspects of the data.

The visual perception of 3D shape

A fundamental problem for the visual perception of 3D shape is that patterns of optical stimulation are inherently ambiguous. Recent mathematical analyses have shown, however, that these ambiguities can be highly constrained, so that many aspects of 3D structure are uniquely specified even though others might be underdetermined. Empirical results with human observers reveal a similar pattern of performance. Judgments about 3D shape are often systematically distorted relative to the actual structure of an observed scene, but these distortions are typically constrained to a limited class of transformations. These findings suggest that the perceptual representation of 3D shape involves a relatively abstract data structure that is based primarily on qualitative properties that can be reliably determined from visual information.

The perception of 3-dimensional affine structure from minimal apparent motion sequences

The research described in the present article was designed to identify the minimal conditions for the visual perception of 3-dimensional structure from motion by comparing the theoretical limitations of ideal observers with the perceptual performance of actual human subjects on a variety of psychophysical tasks. The research began with a mathematical analysis, which showed that 2-frame apparent motion sequences are theoretically sufficient to distinguish between rigid and nonrigid motion and to identify structural properties of an object that remain invariant under affine transformations, but that 3 or more distinct frames are theoretically necessary to adequately specify properties of euclidean structure such as the relative 3-dimensional lengths or angles between nonparallel line segments. A series of four experiments was then performed to verify the psychological validity of this analysis. The results demonstrated that the determination of structure from motion in actual human observers may be restricted to the use of first order temporal relations, which are available within 2-frame apparent motion sequences. That is to say, the accuracy of observers’ judgments did not improve in any of these experiments as the number of distinct frames in an apparent motion sequence was increased from 2 to 8, and performance on tasks involving affine structure was of an order of magnitude greater than performance on similar tasks involving euclidean structure.

Similarities and differences in motion processing between the human and macaque brain: evidence from fMRI.

The present report reviews a series of functional magnetic resonance imaging (fMRI) activation studies conducted in parallel in awake monkeys and humans using the same motion stimuli in both species. These studies reveal that motion stimuli engage largely similar cortical regions in the two species. These common regions include MT/V5 and its satellites, of which FST contributes more to the human motion complex than is generally assumed in human imaging. These results also establish a direct link between selectivity of MT/V5 neurons for speed gradients and functional activation of human MT/V5 by three-dimensional (3D) structure from motion stimuli. On the other hand, striking functional differences also emerged: in humans V3A and several regions in the intraparietal sulcus (IPS) are much more motion sensitive than their simian counterparts.

Perception of surface curvature and direction of illumination from patterns of shading

Three experiments examine the perceptual salience of shading information for the visual specification of three-dimensional form. The observers in these experiments were required to estimate the surface curvature and direction of illumination depicted in computer-synthesized images of cylindrical surfaces, both with and without texture. The results indicate that the shininess of a surface enhances the perception of curvature, but has no effect on perceived direction of illumination; and that shading is generally less effective than texture for depicting surfaces in three dimensions. These and other findings are used to evaluate the psychological validity of several mathematical analyses of shading information that have recently been proposed in the literature.

Human cortical regions involved in extracting depth from motion.

We used functional magnetic resonance imaging (fMRI) to investigate brain regions involved in extracting three-dimensional structure from motion. A factorial design included two-dimensional and three-dimensional structures undergoing rigid and nonrigid motions. As predicted from monkey data, the human homolog of MT/V5 was significantly more active when subjects viewed three-dimensional (as opposed to two-dimensional) displays, irrespective of their rigidity. Human MT/V5+ (hMT/V5+) is part of a network with right hemisphere dominance involved in extracting depth from motion, including a lateral occipital region, five sites along the intraparietal sulcus (IPS), and two ventral occipital regions. Control experiments confirmed that this pattern of activation is most strongly correlated with perceived three-dimensional structure, in as much as it arises from motion and cannot be attributed to numerous two-dimensional image properties or to saliency.

The processing of three-dimensional shape from disparity in the human brain.

Three-dimensional (3D) shape is important for the visual control of grasping and manipulation and for object recognition. Although there has been some progress in our understanding of how 3D shape is extracted from motion and other monocular cues, little is known of how the human brain extracts 3D shape from disparity, commonly regarded as the strongest depth cue. Previous fMRI studies in the awake monkey have established that the interaction between stereo (present or absent) and the order of disparity (zero or second order) constitutes the MR signature of regions housing second-order disparity-selective neurons (Janssen et al., 2000; Srivastava et al., 2006; Durand et al., 2007; Joly et al., 2007). Testing the interaction between stereo and order of disparity in a large cohort of human subjects, revealed the involvement of five IPS regions (VIPS/V7*, POIPS, DIPSM, DIPSA, and phAIP), as well as V3 and the V3A complex in occipital cortex, the posterior inferior temporal gyrus (ITG), and ventral premotor cortex (vPrCS) in the extraction and processing of 3D shape from stereo. Control experiments ruled out attention and convergence eye movements as confounding factors. Many of these regions, DIPSM, DIPSA, phAIP, and probably posterior ITG and ventral premotor cortex, correspond to monkey regions with similar functionality, whereas the evolutionarily new or modified regions are located in occipital (the V3A complex) and occipitoparietal cortex (VIPS/V7* and POIPS). Interestingly, activity in these occipital regions correlates with the depth amplitude perceived by the subjects in the 3D surfaces used as stimuli in these fMRI experiments.

Anterior Regions of Monkey Parietal Cortex Process Visual 3D Shape

The intraparietal cortex is involved in the control of visually guided actions, like reach-to-grasp movements, which require extracting the 3D shape and position of objects from 2D retinal images. Using fMRI in behaving monkeys, we investigated the role of the intraparietal cortex in processing stereoscopic information for recovering the depth structure and the position in depth of objects. We found that while several areas (CIP, LIP, and AIP on the lateral bank; PIP and MIP on the medial bank) are activated by stereoscopic stimuli, AIP and an adjoining portion of LIP are sensitive only to depth structure. Furthermore, only these two regions are sensitive to both the depth structure and the 2D shape of small objects. These results indicate that extracting 3D spatial information from stereo involves several intraparietal areas, among which AIP and anterior LIP are more specifically engaged in extracting the 3D shape of objects.

The visual perception of smoothly curved surfaces from minimal apparent motion sequences

A series of four experiments was designed to investigate the minimal amounts of information required to perceive the structure of a smoothly curved surface from its pattern of projected motion. In Experiments 1 and 2, observers estimated the amplitudes of sinusoidally corrugated surfaces relative to their periods. Observers’ judgments varied linearly with the depicted surface amplitudes, but the amount of perceived relative depth was systematically overestimated by approximately 30%. The observers’ amplitude judgments were also influenced to a lesser extent by the amount of rotary displacement of a surface at each frame transition, and by increasing the length of the apparent motion sequences from two to eight frames. The latter effect of sequence length was quite small, however, accounting for less than 3% of the variance in the observers’ judgments. Experiments 3 and 4 examined observers’ discrimination thresholds for sinusoidally corrugated surfaces of variable amplitude and for ellipsoid surfaces of variable eccentricity. The results revealed that observers could reliably detect differences of surface structure as small as 5%. The length of the apparent motion sequences had no detectable effect on these tasks, although there were significant effects of angular displacement and surface orientation. These results are considered with respect to the analysis of affine structure from motion proposed by Todd and Bressan (1990).

Implications of a transient-sustained dichotomy for the measurement of human performance

Recent evidence suggests that the human visual system has two components: a sustained system that will respond to static contrasts and a transient system that will only respond to rapid changes over time. The present article provides further support for a transient-sustained dichotomy of visual information processing by examining the effects of abrupt changes in visual stimulation in a variety of situations. Several experiments are reported in which stimuli are presented both with and without abrupt onsets. The results of these experiments, together with other evidence, suggest that the overall effects of abrupt changes in visual stimulation may be more extensive than has previously been suspected.