Laurence T. Maloney

Measurement and Modeling of Depth Cue Combination: in Defense of Weak Fusion

Various visual cues provide information about depth and shape in a scene. When several of these cues are simultaneously available in a single location in the scene, the visual system attempts to combine them. In this paper, we discuss three key issues relevant to the experimental analysis of depth cue combination in human vision: cue promotion, dynamic weighting of cues, and robustness of cue combination. We review recent psychophysical studies of human depth cue combination in light of these issues. We organize the discussion and review as the development of a model of the depth cue combination process termed modified weak fusion (MWF). We relate the MWF framework to Bayesian theories of cue combination. We argue that the MWF model is consistent with previous experimental results and is a parsimonious summary of these results. While the MWF model is motivated by normative considerations, it is primarily intended to guide experimental analysis of depth cue combination in human vision. We describe experimental methods, analogous to perturbation analysis, that permit us to analyze depth cue combination in novel ways. In particular these methods allow us to investigate the key issues we have raised. We summarize recent experimental tests of the MWF framework that use these methods.

Color constancy: a method for recovering surface spectral reflectance.

Human and machine visual sensing is enhanced when surface properties of objects in scenes, including color, can be reliably estimated despite changes in the ambient lighting conditions. We describe a computational method for estimating surface spectral reflectance when the spectral power distribution of the ambient light is not known.

Ubiquitous log odds: a common representation of probability and frequency distortion in perception, action, and cognition

In decision from experience, the source of probability information affects how probability is distorted in the decision task. Understanding how and why probability is distorted is a key issue in understanding the peculiar character of experience-based decision. We consider how probability information is used not just in decision-making but also in a wide variety of cognitive, perceptual, and motor tasks. Very similar patterns of distortion of probability/frequency information have been found in visual frequency estimation, frequency estimation based on memory, signal detection theory, and in the use of probability information in decision-making under risk and uncertainty. We show that distortion of probability in all cases is well captured as linear transformations of the log odds of frequency and/or probability, a model with a slope parameter, and an intercept parameter. We then consider how task and experience influence these two parameters and the resulting distortion of probability. We review how the probability distortions change in systematic ways with task and report three experiments on frequency distortion where the distortions change systematically in the same task. We found that the slope of frequency distortions decreases with the sample size, which is echoed by findings in decision from experience. We review previous models of the representation of uncertainty and find that none can account for the empirical findings.

Statistical decision theory and the selection of rapid, goal-directed movements

We present two experiments that test the range of applicability of a movement planning model (MEGaMove) based on statistical decision theory. Subjects attempted to earn money by rapidly touching a green target region on a computer screen while avoiding nearby red penalty regions. In two experiments we varied the magnitudes of penalties, the degree of overlap of target and penalty regions, and the number of penalty regions. Overall, subjects acted so as to maximize gain in a wide variety of stimulus configurations, in good agreement with predictions of the model.

A perturbation analysis of depth perception from combinations of texture and motion cues.

We examined how depth information from two different cue types (object motion and texture gradient) is integrated into a single estimate in human vision. Two critical assumptions of a recent model of depth cue combination (termed modified weak fusion) were tested. The first assumption is that the overall depth estimate is a weighted linear combination of the estimates derived from the individual cues, after initial processing needed to bring them to a common format. The second assumption is that the weight assigned to a cue reflects the apparent reliability of that cue in a particular scene. By this account, the depth combination rule is linear and dynamic, changing in a predictable fashion in response to the particular scene and viewing conditions. A novel procedure was used to measure the weights assigned to the texture and motion cues across experimental conditions. This procedure uses a type of perturbation analysis. The results are consistent with the weighted linear combination rule. In addition, when either cue is corrupted by added noise, the weighted linear combination rule shifts in favor of the uncontaminated cue.

Decision Making, Movement Planning, and Statistical Decision Theory

We discuss behavioral studies directed at understanding how probability information is represented in motor and economic tasks. By formulating the behavioral tasks in the language of statistical decision theory, we can compare performance in equivalent tasks in different domains. Subjects in traditional economic decision-making tasks often misrepresent the probability of rare events and typically fail to maximize expected gain. By contrast, subjects in mathematically equivalent movement tasks often choose movement strategies that come close to maximizing expected gain. We discuss the implications of these different outcomes, noting the evident differences between the source of uncertainty and how information about uncertainty is acquired in motor and economic tasks.

Statistical decision theory and trade-offs in the control of motor response

We present a novel approach to the modeling of motor responses based on statistical decision theory. We begin with the hypothesis that subjects are ideal motion planners who choose movement trajectories to minimize expected loss. We derive predictions of the hypothesis for movement in environments where contact with specified regions carries rewards or penalties. The model predicts shifts in a subjects aiming point in response to changes in the reward and penalty structure of the environment and with changes in the subjects uncertainty in carrying out planned movements. We tested some of these predictions in an experiment where subjects were rewarded if they succeeded in touching a target region on a computer screen within a specified time limit. Near the target was a penalty region which, if touched, resulted in a penalty. We varied distance between the penalty region and the target and the cost of hitting the penalty region. Subjects shift their mean points of contact with the computer screen in response to changes in penalties and location of the penalty region relative to the target region in qualitative agreement with the predictions of the hypothesis. Thus, movement planning takes into account extrinsic costs and the subjects own motor uncertainty.

Conjoint Measurement of Gloss and Surface Texture

The image of a materials surface varies not only with viewing and illumination conditions, but also with the materials surface properties, including its 3-D texture and specularity. Previous studies on the visual perception of surface material have typically focused on single material properties, ignoring possible interactions. In this study, we used a conjoint-measurement design to determine how observers represent perceived 3-D texture (“bumpiness”) and specularity (“glossiness”) and modeled how each of these two surface-material properties affects perception of the other. Observers made judgments of bumpiness and glossiness of surfaces that varied in both surface texture and specularity. We quantified how changes in each surface-material property affected judgments of the other and found that a simple additive model captured visual perception of texture and specularity and their interaction. Conjoint measurement is potentially a powerful tool for analyzing perception of surface material in realistic environments.

Maximum likelihood difference scaling

We present a stochastic model of suprathreshold perceptual differences based on difference measurement. We develop a maximum likelihood difference scaling (MLDS) method for estimating its parameters and evaluate the reliability and distributional robustness of the fitting method. We also describe a method for testing whether the difference measurement model is appropriate as a description of human judgment of perceptual differences in any specific experimental context.

A Statistical Framework for Robust Fusion of Depth Information

We describe a simple statistical framework intended as a model of how depth estimates derived from consistent depth cues are combined in biological vision. We assume that the rule of combination is linear, and that the weights assigned to estimates in the linear combination are variable. These weight values corresponding to different depth cues are determined by ancillary measures, information concerning the likely validity of different depth cues in a particular scene. The parameters of the framework may be estimated psychophysically by procedures described. The conditions under which the framework may be regarded as normative are discussed.