David C. Knill

Visual Feedback Control of Hand Movements

We investigated what visual information contributes to on-line control of hand movements. It has been suggested that motion information predominates early in movements but that position information predominates for endpoint control. We used a perturbation method to determine the relative contributions of motion and position information to feedback control. Subjects reached to touch targets in a dynamic virtual environment in which subjects viewed a moving virtual fingertip in place of their own finger. On some trials, we perturbed the virtual fingertip while it moved behind an occluder. Subjects responded to perturbations that selectively altered either motion or position information, indicating that both contribute to feedback control. Responses to perturbations that changed both motion and position information were consistent with superimposed motion-based and position-based control. Results were well fit by a control model that optimally integrates noisy, delayed sensory feedback about both motion and position to estimate hand state.

Human discrimination of fractal images

In order to transmit information in images efficiently, the visual system should be tuned to the statistical structure of the ensemble of images that it sees. Several authors have suggested that the ensemble of natural images exhibits fractal behavior and, therefore, has a power spectrum that drops off proportionally to 1/f beta (2 less than beta less than 4). In this paper we investigate the question of which value of the exponent beta describes the power spectrum of the ensemble of images to which the visual system is optimally tuned. An experiment in which subjects were asked to discriminate randomly generated noise textures based on their spectral drop-off was used. Whereas the discrimination-threshold function of an ideal observer was flat for different spectral drop-offs, human observers showed a broad peak in sensitivity for 2.8 less than beta less than 3.6. The results are consistent with, but do not provide direct evidence for, the theory that the visual system is tuned to an ensemble of images with Markov statistics.

The perception of cast shadows

When an object casts its shadow on a background surface, the shadow can be informative about the shape of the object, the shape of the background surface and the spatial arrangement of the object relative to the background. Among all these roles, we found that cast shadows were perceptually most relevant for the recovery of spatial arrangement, especially when the shadow is in motion. This finding is intriguing when one considers the ambiguities in the possible ways that shadow motion can be interpreted. We reasoned that the visual system must use a priori constraints to disambiguate the cast shadow motion. One of these constraints is that the light source is stationary. Though simple, the stationary-light-source constraint supports rich, reliable inferences about the qualitative motions of objects in three dimensions.

Moving Cast Shadows Induce Apparent Motion in Depth

Phenomenally strong visual illusions are described in which the motion of an object‘s cast shadow determines the perceived 3-D trajectory of the object. Simply adjusting the motion of a shadow is sufficient to induce dramatically different apparent trajectories of the object casting the shadow. Psychophysical results obtained with the use of 3-D graphics are reported which show that: (i) the information provided by the motion of an objects shadow overrides other strong sources of information and perceptual biases, such as the assumption of constant object size and a general viewpoint; (ii) the natural constraint of shadow darkness plays a role in the interpretation of a moving image patch as a shadow, but under some conditions even unnatural light shadows can induce apparent motion in depth of an object; (iii) when shadow motion is caused by a moving light source, the visual system incorrectly interprets the shadow motion as consistent with a moving object, rather than a moving light source. The results support the hypothesis that the human visual system incorporates a stationary light-source constraint in the perceptual processing of spatial layout of scenes.

Object classification for human and ideal observers

We describe a novel approach, based on ideal observer analysis, for measuring the ability of human observers to use image information for 3D object perception. We compute the statistical efficiency of subjects relative to an ideal observer for a 3D object classification task. After training to 11 different views of a randomly shaped thick wire object, subjects were asked which of a pair of noisy views of the object best matched the learned object. Efficiency relative to the actual information in the stimuli can be as high as 20%. Increases in object regularity (e.g. symmetry) lead to increases in the efficiency with which novel views of an object could be classified. Furthermore, such increases in regularity also lead to decreases in the effect of viewpoint on classification efficiency. Human statistical efficiencies relative to a 2D ideal observer exceeded 100%, thereby excluding all models which are sub-optimal relative to the 2D ideal.

Humans use continuous visual feedback from the hand to control both the direction and distance of pointing movements

Vision of the hand during reaching provides dynamic feedback that can be used to control movement. We investigated the relative contributions of feedback about the direction and distance of the hand relative to a target. Subjects made pointing movements in a 3-D virtual environment, in which a small sphere provided dynamic visual feedback about the position of their unseen fingertip. On a subset of trials, the position of the virtual fingertip was smoothly shifted by 2 cm during movement, either (1) in the direction of movement, which would require adjustments to the distance moved, or (2) orthogonal to the direction of movement, which would require adjustments to the direction moved. Despite not noticing the perturbations, subjects adjusted their movements to compensate for both types of visual shifts. Corrective responses to direction perturbations were observed within 117 ms, and response latencies were invariant to movement speed and perturbation onset time. Initial corrections to distance perturbations were smaller and appeared after longer delays of 130–200 ms, and both the speed and magnitude of responses were reduced for early onset perturbations. Simulations of a feedback control model that optimally integrates visual information over time show that the results can be explained by differences in the sensory noise levels in the visual dimensions relevant for direction and distance control.

Bayesian sampling in visual perception

It is well-established that some aspects of perception and action can be understood as probabilistic inferences over underlying probability distributions. In some situations, it would be advantageous for the nervous system to sample interpretations from a probability distribution rather than commit to a particular interpretation. In this study, we asked whether visual percepts correspond to samples from the probability distribution over image interpretations, a form of sampling that we refer to as Bayesian sampling. To test this idea, we manipulated pairs of sensory cues in a bistable display consisting of two superimposed moving drifting gratings, and we asked subjects to report their perceived changes in depth ordering. We report that the fractions of dominance of each percept follow the multiplicative rule predicted by Bayesian sampling. Furthermore, we show that attractor neural networks can sample probability distributions if input currents add linearly and encode probability distributions with probabilistic population codes.

Cognition : conceptual and methodological issues

Raises conceptual and metatheoretical questions against a background of relevant empirical investigations, in 13 papers originally presented at a conference organized by the U. of Minnesota Center for Research in Learning, Perception, and Cognition, and held February-March 1991. The papers are group

Mixture models and the probabilistic structure of depth cues

Monocular cues to depth derive their informativeness from a combination of perspective projection and prior constraints on the way scenes in the world are structured. For many cues, the appropriate priors are best described as mixture models, each of which characterizes a different category of objects, surfaces, or scenes. This paper provides a Bayesian analysis of the resulting model selection problem, showing how the mixed structure of priors creates the potential for non-linear, cooperative interactions between cues and how the information provided by a single cue can effectively determine the appropriate constraint to apply to a given image. The analysis also leads to a number of psychophysically testable predictions. We test these predictions by applying the framework to the problem of perceiving planar surface orientation from texture. A number of psychophysical experiments are described that show that the visual system is biased to interpret textures as isotropic, but that when sufficient image data is available, the system effectively turns off the isotropy constraint and interprets texture information using only a homogeneity assumption. Human performance is qualitatively similar to an optimal estimator that assumes a mixed prior on surface textures--some proportion being isotropic and homogeneous and some proportion being anisotropic and homogeneous.

Stereopsis and amblyopia: A mini-review

Amblyopia is a neuro-developmental disorder of the visual cortex that arises from abnormal visual experience early in life. Amblyopia is clinically important because it is a major cause of vision loss in infants and young children. Amblyopia is also of basic interest because it reflects the neural impairment that occurs when normal visual development is disrupted. Amblyopia provides an ideal model for understanding when and how brain plasticity may be harnessed for recovery of function. Over the past two decades there has been a rekindling of interest in developing more effective methods for treating amblyopia, and for extending the treatment beyond the critical period, as exemplified by new clinical trials and new basic research studies. The focus of this review is on stereopsis and its potential for recovery. Impaired stereoscopic depth perception is the most common deficit associated with amblyopia under ordinary (binocular) viewing conditions (Webber & Wood, 2005). Our review of the extant literature suggests that this impairment may have a substantial impact on visuomotor tasks, difficulties in playing sports in children and locomoting safely in older adults. Furthermore, impaired stereopsis may also limit career options for amblyopes. Finally, stereopsis is more impacted in strabismic than in anisometropic amblyopia. Our review of the various approaches to treating amblyopia (patching, perceptual learning, videogames) suggests that there are several promising new approaches to recovering stereopsis in both anisometropic and strabismic amblyopes. However, recovery of stereoacuity may require more active treatment in strabismic than in anisometropic amblyopia. Individuals with strabismic amblyopia have a very low probability of improvement with monocular training; however they fare better with dichoptic training than with monocular training, and even better with direct stereo training.