Phillip Marlow

The Perception and Misperception of Specular Surface Reflectance

The amount and spectral content of the light reflected by most natural surfaces depends on the structure of the light field, the observers viewing position, and 3D surface geometry, particularly for specular (glossy) surfaces. A growing body of data has demonstrated that perceived surface gloss can vary as a function of its 3D shape and its illumination field, but there is currently no explanation for these effects. Here, we show that the perception of gloss can be understood as a direct consequence of image properties that covary with surface geometry and the illumination field. We show that different illumination fields can generate qualitatively different patterns of interaction between perceived gloss and 3D surface geometry. Despite the complexity and variability of these interactions, we demonstrate that the perception (and misperception) of gloss is well predicted by the way that each illumination field modulates the size, contrast, sharpness, and depth of specular reflections. Our results provide a coherent explanation of the effects of extrinsic scene variables on perceived gloss, and our methods suggest a general technique for assessing the role of specific image properties in modulating our visual experience of material properties.

The perception of gloss depends on highlight congruence with surface shading

Studies have shown that displacing specular highlights from their natural locations in images reduces perceived surface gloss. Here, we assessed the extent to which perceived gloss depends on congruence in the position and orientation of specular highlights relative to surface shape and the diffuse shading from which surface shape is recovered. The position and orientation congruence of specular highlights with diffuse shading was altered while preserving their compatibility with physical surface shape (Experiment 1). We found that perceived gloss diminished as the position of highlights became incompatible wit h the surfaces global diffuse shading maxima. In a subsequent experiment, we constrained highlight proximity near the global luminance maxima in diffuse shading. When we disrupted the consistency in the local position and orientation of specular highlights with respect to the diffuse shading and local surface meso-structure, a decline in perceived gloss was still observed (Experiment 2). This decline in perceived gloss caused by misaligning the positions and orientations of specular highlights relative to diffuse surface shading could not be explained by differences in orientation fields alone (Experiments 3 and 4). These results suggest the visual system assesses both position and orientation congruence between specular highlights and diffuse shading to estimate surface gloss.

The role of brightness and orientation congruence in the perception of surface gloss

The perception of surface gloss depends on specular highlights but little is understood about how the visual system distinguishes specular highlights from other luminance maxima generated by variations in pigmentation or illumination. It has been argued that diffuse shading gradients provide information for identifying specular highlights. Specular highlights typically share the orientation of the diffuse shading locally. Specular highlights are typically proximal to the brightest region of the diffuse shading locally. We compared the contributions of these two relationships to perceived gloss. Highlight orientation relative to the diffuse shading was varied by rotating highlights. Highlight distance from the brightest region of the diffuse shading was varied by translating highlights in displays that preserved the orientations of highlights relative to their surrounds. Both manipulations reduced perceived gloss. Rotations reduced perceived gloss more than translations, even though translations displaced highlights into darker regions than rotations. The same reductions in perceived gloss occurred when highlights were matched in perceived contrast across conditions (Experiment 2b). The results provide evidence that the perception of gloss depends on highlight distance from the luminance maxima of the surrounding intensity gradient (brightness congruence) in addition to the shared orientation of highlights with their surrounds (orientation congruence).

Texture-shading flow interactions and perceived reflectance.

The appearance of surface texture depends on the identification of edge contours in an image generated by local variations in reflectance. These edges in the image need to be distinguished from diffuse shading gradients caused by the interaction of light with surface relief. To understand how the brain performs this separation, we generated textures with orientation flows that were initially congruent with the diffuse shading flow of planar surfaces. We found that rotating textures relative to shading increased the appearance of surface pigmentation, which was well explained by an increase in the variation of local orientation fields with increasing offset of texture gradients (Experiment 1). We obtained similar findings when rotating texture flow relative to the diffuse shading of spherical surfaces with global curvature (Experiment 2). In a second set of experiments, we found that perceived pigmentation of spherical surfaces depended on the perceived orientation of the light field; rotating images of spherical surfaces reduced both perceived pigmentation (Experiment 3) and perceived global texture contrast in an objective task (Experiment 4). The dependence of perceived texture on image orientation suggests that the separation of texture flow from shading depends on an assumed light source from above bias. These findings support the view that separation of texture flow from shading, and thus perceived pigmentation, depend not only on the local structure of orientation fields in an image, but also on midlevel representations of shading and illuminance flow.

Coupled computations of three-dimensional shape and material

Retinal image structure arises from the interaction between a surfaces three-dimensional shape, its reflectance and transmittance properties, and the surrounding light field. Any local image structure can be generated by an infinite number of different combinations of surface properties, which suggests that the visual system must somehow constrain the possible scene interpretations. The research on this has searched for such constraints in statistical regularities of two-dimensional image structure [1,2]. Here, we present a new class of displays in which the perception of material properties cannot be explained with two-dimensional image properties. The displays manipulate the perceived three-dimensional shape of identical luminance gratings, and demonstrate that perceived three-dimensional shape can alter perceived surface reflectance.

Motion and texture shape cues modulate perceived material properties.

Specular and matte surfaces can project identical images if the surface geometry and light field are appropriately configured. Our previous work has shown that the visual system can exploit stereopsis and contour cues to 3D shape to disambiguate different surface reflectance interpretations. Here, we test whether material perception depends on information about surface geometry provided by structure from motion and shape from texture. Different surface textures were superimposed on a fixed pattern of luminance gradients to generate two different 3D shape interpretations. Each shape interpretation of the luminance gradients promoted a different experience of surface reflectance and illumination direction, which varied from a specular surface in frontal illumination to a comparatively matte surface in grazing illumination. The shape that appeared most specular exhibited the steepest derivatives of luminance with respect to surface orientation, consistent with physical differences between specular and diffuse reflectance. The effect of apparent shape on perceived reflectance occurred for a variety of surface textures that provided either structure from motion, shape from texture, or both optical sources of shape information. In conjunction with previous findings (Marlow, Todorović, & Anderson, 2015; Marlow & Anderson, 2015), these results suggest that any cue that provides sufficient information about 3D shape can also be used to derive material properties from the rate that luminance varies as a function of surface curvature.

Local and non-local effects on surface-mediated stereoscopic depth

The magnitude and precision of stereoscopic depth between two probes is often determined by the disparity each has to a common background. If stereoscopic slant of the background is underestimated, a bias is introduced in the PSE of the probes (G. Mitchison & G. Westheimer, 1984). Using random dot stimuli, we show here how more remote surfaces can influence probe PSE via their influence on perceived background surface slant. The bias was reduced when frontal flanking surfaces were placed above and below the background surface, increasing its perceived slant. In a similar experiment, the flankers were slanted and the central background surface was frontal. For flankers alone, probe bias did not diminish up to a 4.4° separation of flankers and probes. When the central surface was present, the effect of the flankers on probe bias was mediated by this surface and diminished with flanker separation, presumably because of the diminishing contrast slant of the background surface. Stereoscopic depth between probes is thus influenced by a common background surface, by neighboring surfaces acting (contiguously or non-contiguously) on the background surface, and by distant surfaces acting directly on the probes.

Turning the World Upside Down to Understand Perceived Transparency

Specular surfaces and refractive media are difficult to distinguish from each other because they both generate distorted images of the surrounding lighting environment. Whereas convex refractive objects invert the orientation of the horizon so the sky appears beneath the ground plane, convex specular surfaces preserve the orientation of the horizon so the sky appears above the ground. Here, we show that a refractive transparent object can be made to appear specular and opaque simply by rotating the image by 180°. This result suggests that the visual system relies on information tied to the orientation of the horizon to distinguish between refractive and specular objects.

Stereopsis Loses Dominance over Relative Size as Target Separation Increases

Binocular disparity produces less stereoscopic depth if the targets are separated by several degrees. It is thus possible that separation decreases the influence of stereopsis as a relative depth cue. Here, four experiments tested the strength of disparity in determining the direction of relative depth in the face of strongly conflicting relative size for a range of target separations. Under conditions of natural fixation—permitting sequential stereopsis—disparity dominated completely at small separations (0.42°) but gradually gave way to relative size domination at large separations. However, when brief presentations prevented changes in fixation, disparity completely dominated at a separation of 0.5° while relative size mostly dominated by 0.75° – 1° of separation. By varying target separation at different retinal eccentricities, we showed that separation per se was the critical factor in the dominance switch. Stereoacuity as a function of target separation for the same observers did not predict the switch from disparity to relative size. Stereoscopic dominance was found for the same small separations that are immune to stereoscopic reversals (Gillam, 1993 Perception 22 1025 – 1036). Our results suggest that relative disparity has a compulsory influence on perceived depth at small separations, suggesting a different mechanism from the one operating at larger separations.

Comparing Subjective Contours for Kanizsa Squares and Linear Edge Alignments (‘New York Titanic’ Figures)

One current view is that subjective contours may involve high-level detection of a salient shape with back propagation to early visual areas where small receptive fields allow for scrutiny of relevant details. This idea applies to Kanizsa-type figures. However, Gillam and Chan (2002 Psychological Science, 13, 279–282) using figures based on Gillams graphic ‘New York Titanic’ (Gillam, 1997 Thresholds: Limits of perception. New York: Arts Magazine) showed that strong subjective contours can be seen along the linearly aligned edges of a set of shapes if occlusion cues of ‘extrinsic edge’ and ‘entropy contrast’ are strong. Here we compared ratings of the strength of subjective contours along linear alignments with those seen in Kanizsa figures. The strongest subjective contour for a single set of linearly aligned shapes was similar in strength to the edges of a Kanizsa square (controlling for support ratio) despite the lack of a salient region. The addition of a second set of linearly aligned inducers consistent with a common surface increased subjective-contour strength, as did having four rather than two ‘pacmen’ in the Kanizsa figure, indicating a role for surface support. We argue that linear subjective contours allow for the investigation of certain occlusion cues and the interactions between them that are not easily explored with Kanizsa figures.