Tandra Ghose

Extremal Edge A Powerful Cue to Depth Perception and Figure-Ground Organization

Extremal edges (EEs) are projections of viewpoint-specific horizons of self-occlusion on smooth convex surfaces. An ecological analysis of viewpoint constraints suggests that an EE surface is likely to be closer to the observer than the non-EE surface on the other side of the edge. In two experiments, one using shading gradients and the other using texture gradients, we demonstrated that EEs operate as strong cues to relative depth perception and figure-ground organization. Image regions with an EE along the shared border were overwhelmingly perceived as closer than either flat or equally convex surfaces without an EE along that border. A further demonstration suggests that EEs are more powerful than classical figure-ground cues, including even the joint effects of small size, convexity, and surroundedness.

Relative image size, not eye position, determines eye dominance switches.

A recent paper examined eye dominance with the eyes in forward and eccentric gaze [Vision Res. 41 (2001) 1743]. When observers were looking to the left, the left eye tended to dominate and when they were looking to the right, the right eye tended to dominate. The authors attributed the switch in eye dominance to extra-retinal signals associated with horizontal eye position. However, when one looks at a near object on the left, the image in the left eye is larger than the one in the right eye, and when one looks to the right, the opposite occurs. Thus, relative image size could also trigger switches in eye dominance. We used a cue-conflict paradigm to determine whether eye position or relative image size is the determinant of eye-dominance switches with changes in gaze angle. When eye position and relative image size were varied independently, there was no consistent effect of eye position. Relative image size appears to be the sole determinant of the switch.

Forensic comparison and matching of fingerprints: using quantitative image measures for estimating error rates through understanding and predicting difficulty.

Latent fingerprint examination is a complex task that, despite advances in image processing, still fundamentally depends on the visual judgments of highly trained human examiners. Fingerprints collected from crime scenes typically contain less information than fingerprints collected under controlled conditions. Specifically, they are often noisy and distorted and may contain only a portion of the total fingerprint area. Expertise in fingerprint comparison, like other forms of perceptual expertise, such as face recognition or aircraft identification, depends on perceptual learning processes that lead to the discovery of features and relations that matter in comparing prints. Relatively little is known about the perceptual processes involved in making comparisons, and even less is known about what characteristics of fingerprint pairs make particular comparisons easy or difficult. We measured expert examiner performance and judgments of difficulty and confidence on a new fingerprint database. We developed a number of quantitative measures of image characteristics and used multiple regression techniques to discover objective predictors of error as well as perceived difficulty and confidence. A number of useful predictors emerged, and these included variables related to image quality metrics, such as intensity and contrast information, as well as measures of information quantity, such as the total fingerprint area. Also included were configural features that fingerprint experts have noted, such as the presence and clarity of global features and fingerprint ridges. Within the constraints of the overall low error rates of experts, a regression model incorporating the derived predictors demonstrated reasonable success in predicting objective difficulty for print pairs, as shown both in goodness of fit measures to the original data set and in a cross validation test. The results indicate the plausibility of using objective image metrics to predict expert performance and subjective assessment of difficulty in fingerprint comparisons.

Extremal edges versus other principles of figure-ground organization

Identifying the visual cues that determine relative depth across an image contour (i.e., figure-ground organization) is a central problem of vision science. In this paper, we compare flat cues to figure-ground organization with the recently discovered cue of extremal edges (EEs), which arise when opaque convex surfaces smoothly curve to partly occlude themselves. The present results show that EEs are very powerful pictorial cues to relative depth across an edge, almost entirely dominating the well-known figure-ground cues of relative size, convexity, shape familiarity, and surroundedness. These results demonstrate that natural shading and texture gradients in an image provide important information about figure-ground organization that has largely been overlooked in the past 75 years of research on this topic.

How the global layout of the mask influences masking strength

In visual backward masking, the perception of a target is influenced by a trailing mask. Masking is usually explained by local interactions between the target and the mask representations. However, recently it has been shown that the global spatial layout of the mask rather than its local structure determines masking strength (Hermens & Herzog, 2007). Here, we varied the mask layout by spatial, luminance, and temporal cues. We presented a vernier target followed by a mask with 25 elements. Performance deteriorated when the length of the two mask elements neighboring the target vernier was doubled. However, when the length of every second mask element was doubled, performance improved. When the luminance of the neighboring elements was doubled, performance also deteriorated but no improvement in performance was observed when every second element had a double luminance. For temporal manipulations, a complex nonmonotonic masking function was observed. Hence, changes in the mask layout by spatial, luminance, and temporal cues lead to highly different results.

IBES : a tool for creating instructions based on event segmentation

Receiving informative, well-structured, and well-designed instructions supports performance and memory in assembly tasks. We describe IBES, a tool with which users can quickly and easily create multimedia, step-by-step instructions by segmenting a video of a task into segments. In a validation study we demonstrate that the step-by-step structure of the visual instructions created by the tool corresponds to the natural event boundaries, which are assessed by event segmentation and are known to play an important role in memory processes. In one part of the study, 20 participants created instructions based on videos of two different scenarios by using the proposed tool. In the other part of the study, 10 and 12 participants respectively segmented videos of the same scenarios yielding event boundaries for coarse and fine events. We found that the visual steps chosen by the participants for creating the instruction manual had corresponding events in the event segmentation. The number of instructional steps was a compromise between the number of fine and coarse events. Our interpretation of results is that the tool picks up on natural human event perception processes of segmenting an ongoing activity into events and enables the convenient transfer into meaningful multimedia instructions for assembly tasks. We discuss the practical application of IBES, for example, creating manuals for differing expertise levels, and give suggestions for research on user-oriented instructional design based on this tool.

Temporal aspects of spatial interactions affecting stereo-matching solutions

Stereo-matching solutions minimize disparity relative to the horopter (minimum-absolute-disparity or MAD), and differences in disparity between adjacent features (minimum-relative-disparity or MRD). When placed in conflict, spatial proximity promotes MRD over MAD solutions. How does temporal proximity of neighboring features affect strength of these spatial interactions? We quantified the inter-stimulus interval (ISI) over which an unambiguous disparity pattern influenced stereo-matches for patterns with several possible solutions. Likelihood of MRD decreased as ISI increased (48.9 ms time constant) and increased as contrast was reduced for short ISIs, suggesting that monocular persistence (temporal impulse response) underlies the temporal interaction.

Gradient cuts and extremal edges in relative depth and figure–ground perception

Extremal edges (EEs) are borders consisting of luminance gradients along the projected edge of a partly self-occluding curved surface (e.g., a cylinder), with equiluminant contours (ELCs) that run approximately parallel to that edge. Gradient cuts (GCs) are similar luminance gradients with ELCs that intersect (are “cut” by) an edge that could be due to occlusion. EEs are strongly biased toward being seen as closer/figural surfaces (Palmer & Ghose, Psychological Science, 19(1), 77-83, 2008). Do GCs produce a complementary bias toward being seen as ground? Experiment 1 shows that, with EEs on the opposite side, GCs produce a ground bias that increases with increasing ELC angles between ELCs and the shared edge. Experiment 2 shows that, with flat surfaces on the opposite side, GCs do not produce a ground bias, suggesting that more than one factor may be operating. We suggest that two partially dissociable factors may operate for curved surfaces—ELC angle and 3-D surface convexity—that reinforce each other in the figural cues of EEs but compete with each other in GCs. Moreover, this figural bias is modulated by the presence of EEs and GCs, as specified by the ELC angle between ELCs and the shared contour.

Recovering metric properties of objects through spatiotemporal interpolation.

Spatiotemporal interpolation (STI) refers to perception of complete objects from fragmentary information across gaps in both space and time. It differs from static interpolation in that requirements for interpolation are not met in any static frame. It has been found that STI produced objective performance advantages in a shape discrimination paradigm for both illusory and occluded objects when contours met conditions of spatiotemporal relatability. Here we report psychophysical studies testing whether spatiotemporal interpolation allows recovery of metric properties of objects. Observers viewed virtual triangles specified only by sequential partial occlusions of background elements by their vertices (the STI condition) and made forced choice judgments of the objects size relative to a reference standard. We found that length could often be accurately recovered for conditions where fragments were relatable and formed illusory triangles. In the first control condition, three moving dots located at the vertices provided the same spatial and timing information as the virtual object in the STI condition but did not induce perception of interpolated contours or a coherent object. In the second control condition oriented line segments were added to the dots and mid-points between the dots in a way that did not induce perception of interpolated contours. Control stimuli did not lead to accurate size judgments. We conclude that spatiotemporal interpolation can produce representations, from fragmentary information, of metric properties in addition to shape.

Low-level pixelated representations suffice for aesthetically pleasing contrast adjustment in photographs

Today’s web-based automatic image enhancement algorithms decide to apply an enhancement operation by searching for “similar” images in an online database of images and then applying the same level of enhancement as the image in the database. Two key bottlenecks in these systems are the storage cost for images and the cost of the search. Based on the principles of computational aesthetics, we consider storing task-relevant aesthetic summaries, a set of features which are sufficient to predict the level at which an image enhancement operation should be performed, instead of the entire image. The empirical question, then, is to ensure that the reduced representation indeed maintains enough information so that the resulting operation is perceived to be aesthetically pleasing to humans. We focus on the contrast adjustment operation, an important image enhancement primitive. We empirically study the efficacy of storing a pixelated summary of the 16 most representative colors of an image and performing contrast adjustments on this representation. We tested two variants of the pixelated image: a “mid-level pixelized version” that retained spatial relationships and allowed for region segmentation and grouping as in the original image and a “low-level pixelized-random version” which only retained the colors by randomly shuffling the 50 x 50 pixels. In an empirical study on 25 human subjects, we demonstrate that the preferred contrast for the low-level pixelized-random image is comparable to the original image even though it retains very few bits and no semantic information, thereby making it ideal for image matching and retrieval for automated contrast editing. In addition, we use an eye tracking study to show that users focus only on a small central portion of the low-level image, thus improving the performance of image search over commonly used computer vision algorithms to determine interesting key points.