Andrea Li

Limitations on shape information provided by texture cues

This paper uses visual, empirical and formal methods (Li & Zaidi, Vision Research, 40 (2000) 217; Li & Zaidi, Vision Research, 41 (22) (2001a) 2927) to examine the roles of oriented texture components in conveying veridical percepts of concave and convex surfaces that are pitched towards or away from the observer. The results show that pairs of components, oriented symmetrically around the axis of maximum curvature, combine to provide the geodesic orientation modulations that are critical for veridical shape perception. The degree of pitch determines the orientations of the critical pair of components. Perspective is crucial to the veridical perception of concavities and convexities, regardless of the degree of pitch. The results of this paper reconfirm that veridical shape perception depends on extracting critical patterns of oriented energy, but also show that the class of textures capable of conveying veridical percepts of developable shapes in general views is even more restricted than that identified by Li and Zaidi (Journal of Optical Society of America A, 18 (2001b), 2430).

Three-dimensional shape from non-homogeneous textures: carved and stretched surfaces.

We examined the perception of 3D shape for surfaces folded, carved, or stretched out of textured materials. The textures were composed of sums of sinusoidal gratings or of circular dots, and were designed to differentiate between orientation and frequency information present in perspective images of the surfaces. Correct perception of concavities, convexities, saddles, and slants required the visibility of signature patterns of orientation modulations. These patterns were identical to those identified previously for developable surfaces (A. Li & Q. Zaidi, 2000; Q. Zaidi & L. Li, 2002), despite the fact that textures were statistically homogeneous on developable surfaces but not on carved or stretched surfaces. Frequency modulations in the image were interpreted as cues to distance from the observer, which led to weak but qualitatively correct percepts for some carved and stretched surfaces but to misperceptions for others, similar to the misperceptions for developable surfaces (A. Li & Q. Zaidi, 2003). Irrespective of whether texture on the surface is homogeneous or non-homogeneous, similar neural modules can be used to locate signature orientation modulations and thus extract shape from texture cues.

Information limitations in perception of shape from texture.

Li and Zaidi (Li, A., and Zaidi, Q. (2000) Vision Research, 40, 217-242) showed that the veridical perception of the 3-dimensional (3D) shape of a corrugated surface from texture cues is entirely dependent on the visibility of critical patterns of oriented energy. These patterns are created by perspective projection of surface markings oriented along lines of maximum 3D curvature. In images missing these orientation modulations, observers confused concavities with convexities, and leftward slants with rightward slants. In this paper, it is shown that these results were a direct consequence of the physical information conveyed by different oriented components of the texture pattern. For texture patterns consisting of single gratings of arbitrary spatial frequency and orientation, equations are derived from perspective geometry that describe the local spatial frequency and orientation for any slant at any height above and below eye level. The analysis shows that only gratings oriented within a few degrees of the axis of maximum curvature exhibit distinct patterns of orientation modulations for convex, concave, and leftward and rightward slanted portions of a corrugated surface. All other gratings exhibit patterns of frequency and orientation modulations that are distinct for curvatures on the one hand and slants on the other, but that are nearly identical for curvatures of different sign, and nearly identical for slants of different direction. The perceived shape of surfaces was measured in a 5AFC paradigm (concave, convex, leftward slant, rightward slant, and flat-frontoparallel). Observers perceived all five shapes correctly only for gratings oriented within a few degrees of the axis of maximum curvature. For all other oriented gratings, observers could distinguish curvatures from slants, but could not distinguish signs of curvature or directions of slant. These results demonstrate that human observers utilize the shape information provided by texture components along both critical and non-critical orientations.

Observer strategies in perception of 3-D shape from isotropic textures: developable surfaces.

We document the limitations of isotropic textures in conveying three-dimensional shape. We measured the perceived shape and pitch of upright and pitched corrugated surfaces overlaid with different classes of isotropic textures: patterns containing isotropic texture elements, isotropically filtered noise patterns, and patterns containing ellipses or lines of all orientations. Frequency modulations arising from surface slant were incorrectly interpreted as changes in surface distance, resulting in concavities being misclassified as convexities, and right and left slants as concavities. In addition, images of pitched surfaces exhibited oriented flows that confound surface shape and surface pitch. Observers related oriented flow patterns to particular surface shapes with a bias for perceiving convex surfaces. When concave and convex curvatures were concurrently visible, the number of correct shape classifications increased slightly. Isotropic textures thus convey correct 3-D shapes of developable surfaces only in some conditions, and the same perceptual strategies lead to non-veridical percepts in other conditions.

Erratum to ''Information limitations in perception of shape from texture'' (Vision Research 41 (2001) 1519-1534)

Li and Zaidi (Li, A., and Zaidi, Q. (2000) Vision Research, 40, 217-242) showed that the veridical perception of the 3-dimensional (3D) shape of a corrugated surface from texture cues is entirely dependent on the visibility of critical patterns of oriented energy. These patterns are created by perspective projection of surface markings oriented along lines of maximum 3D curvature. In images missing these orientation modulations, observers confused concavities with convexities, and leftward slants with rightward slants. In this paper, it is shown that these results were a direct consequence of the physical information conveyed by different oriented components of the texture pattern. For texture patterns consisting of single gratings of arbitrary spatial frequency and orientation, equations are derived from perspective geometry that describe the local spatial frequency and orientation for any slant at any height above and below eye level. The analysis shows that only gratings oriented within a few degrees of the axis of maximum curvature exhibit distinct patterns of orientation modulations for convex, concave, and leftward and rightward slanted portions of a corrugated surface. All other gratings exhibit patterns of frequency and orientation modulations that are distinct for curvatures on the one hand and slants on the other, but that are nearly identical for curvatures of different sign, and nearly identical for slants of different direction. The perceived shape of surfaces was measured in a 5AFC paradigm (concave, convex, leftward slant, rightward slant, and flat-frontoparallel). Observers perceived all five shapes correctly only for gratings oriented within a few degrees of the axis of maximum curvature. For all other oriented gratings, observers could distinguish curvatures from slants, but could not distinguish signs of curvature or directions of slant. These results demonstrate that human observers utilize the shape information provided by texture components along both critical and non-critical orientations.

Veridicality of three-dimensional shape perception predicted from amplitude spectra of natural textures.

We show that the amplitude spectrum of a texture pattern, regardless of its phase spectrum, can be used to predict whether the pattern will convey the veridical three-dimensional (3-D) shape of the surface on which it lies. Patterns from the Brodatz collection of natural textures were overlaid on a flat surface that was then corrugated in depth and projected in perspective. Perceived ordinal shapes, reconstructed from a series of local relative depth judgments, showed that only about a third of the patterns conveyed veridical shape. The phase structure of each pattern was then randomized. Simulated concavities and convexities were presented for both the Brodatz and the phase-randomized patterns in a global shape identification task. The concordance between the shapes perceived from the Brodatz patterns and their phase-randomized versions was 80-88%, showing that the capacity for a pattern to correctly convey concavities and convexities is independent of phase information and that the amplitude spectrum contains all the information required to determine whether a pattern will convey veridical 3-D shape. A measure of the discrete oriented energy centered on the axis of maximum curvature was successful in identifying textures that convey veridical shape.

Perception of 3D shape from homogeneous and nonhomogeneous surface textures

When a textured 3-dimensional surface is projected in perspective, the statistics of the texture in the image change with the shape of the surface. Most shape-from-texture models assume that these changes are due solely to the projection of non-fronto-parallel portions of the surface. This is true for developable surfaces, which are formed by bending or curving flat, textured sheets without tearing or stretching. However, for other surfaces such as those carved from solids or formed by stretched materials, the texture on the surface is generally not homogenous. If the perspective image is parsed into local Fourier spectra, we find that signature patterns of orientation flows occur at locations corresponding to specific 3-D shapes. These patterns occur generically for developable, carved and stretched surfaces and when they are visible, observers make veridical shape judgments. In contrast, frequency modulations vary systematically for different types of surfaces, and often lead to non-veridical percepts when they are caused by changes in slant (e.g. isotropically textured developable surfaces). Our results suggest that in the extraction of 3-D shape, the visual system can generically employ a limited number of neural mechanisms to extract the signature orientation flows from the image regardless of homogeneity.