Benjamin T. Backus

Learning illumination- and orientation-invariant representations of objects through temporal association

As the orientation or illumination of an object changes so does its appearance. This paper considers how observers are nonetheless able to recognize objects that have undergone such changes. In particular the paper tests the hypothesis that observers rely on temporal correlations between different object views to decide whether they are views of the same object or not. In a series of experiments subjects were shown a sequence of views representing a slowly transforming object. Testing revealed that subjects had formed object representations which were directly influenced by the temporal characteristics of the training views. In particular, introducing spurious correlations between views of different peoples heads caused subjects to regard those views as being of a single person. This rapid and robust overriding of basic generalization processes supports the view that our recognition system tracks the correlated appearance of views of objects across time. Such view associations appear to allow the visual system to solve the view invariance problem without recourse to complex illumination models for extracting 3D form, or the use of the image plane transformations required to make appearance-based comparisons.

Genetic and environmental contributions to strabismus and phoria: evidence from twins

The causes of manifest (strabismus) and latent (phoria) misalignment of the visual axes are incompletely understood. We calculated genetic and environmental contributions to strabismus based upon a critical review and quantitative meta-analysis of previous strabismus twin studies (n=3418 twin pairs) and calculated contributions to phoria based upon a new twin study (n=307 twin pairs). Our results suggest that genetic liability is necessary to develop strabismus, whereas environmental factors are sufficient to cause most phorias. The different etiologies implied by this work suggest that strabismus and phoria should be carefully distinguished in epidemiological work.

Uninformative visual experience establishes long term perceptual bias.

Visual appearance depends upon the resolution of ambiguities that arise when 2D retinal images are interpreted as 3D scenes. This resolution may be characterized as a form of Bayesian perceptual inference, whereby retinal sense data combine with prior belief to yield an interpretation. Under this framework, the prior reflects environmental statistics, so an efficient system should learn by changing its prior after exposure to new statistics. We conjectured that a prior would only be modified when sense data contain disambiguating information, such that it is clear what bias is appropriate. This conjecture was tested by using a perceptually bistable stimulus, a rotating wire-frame cube, as a sensitive indicator of changes in the prior for 3D rotation direction, and by carefully matching perceptual experience of ambiguous and unambiguous versions of the stimulus across three groups of observers. We show for the first time that changes in the prior-observed as a change in bias that resists reverse learning the next day-is affected more by ambiguous stimuli than by disambiguated stimuli. Thus, contrary to our conjecture, modification of the prior occurred preferentially when the observer actively resolved ambiguity rather than when the observer was exposed to environmental contingencies. We propose that resolving stimuli that are not easily interpreted by existing visual rules must be a valid method for establishing useful perceptual biases in the natural world.

Absence of Cue-Recruitment for Extrinsic Signals: Sounds, Spots, and Swirling Dots Fail to Influence Perceived 3D Rotation Direction after Training

The visual system can learn to use information in new ways to construct appearance. Thus, signals such as the location or translation direction of an ambiguously rotating wire frame cube, which are normally uninformative, can be learned as cues to determine the rotation direction [1]. This perceptual learning occurs when the formerly uninformative signal is statistically associated with long-trusted visual cues (such as binocular disparity) that disambiguate appearance during training. In previous demonstrations, the newly learned cue was intrinsic to the perceived object, in that the signal was conveyed by the same image elements as the object itself. Here we used extrinsic new signals and observed no learning. We correlated three new signals with long-trusted cues in the rotating cube paradigm: one crossmodal (an auditory signal) and two within modality (visual). Cue recruitment did not occur in any of these conditions, either in single sessions or in ten sessions across as many days. These results suggest that the intrinsic/extrinsic distinction is important for the perceptual system in determining whether it can learn and use new information from the environment to construct appearance. Extrinsic cues do have perceptual effects (e.g. the “bounce-pass” illusion [2] and McGurk effect [3]), so we speculate that extrinsic signals must be recruited for perception, but only if certain conditions are met. These conditions might specify the age of the observer, the strength of the long-trusted cues, or the amount of exposure to the correlation.

Self-reported Magic Eye™ stereogram skill predicts stereoacuity

Autostereograms—commonly known as Magic Eye™ stereograms (MESs)—are two-dimensional images that support stereoscopic depth perception given an appropriate crossing or uncrossing of the eyes. We find that self-reported MES skill is highly predictive of stereoacuity as measured by a standard clinical test (r142 = 0.45, p < 0.0001; TNO test). Indeed, in our sample of 194 individuals, those who report poor MES skill have a five-fold increased risk of stereo impairment. Those who report poor MES skill also require on average five times greater binocular disparity to perceive stereoscopic depth than those who report good MES skill. Reported MES skill thus carries significant information about stereoacuity.

Associative learning of shape as a cue to appearance: A new demonstration of cue recruitment

The perceived rotation direction of a wire-frame Necker cube at stimulus onset can be conditioned to be dependent on retinal location (B. T. Backus & Q. Haijiang, 2007; S. J. Harrison & B. T. Backus, 2010a). This phenomenon was proposed to be an example of the visual system learning new cues to visual appearance, by adaptation in response to new experiences. Here, we demonstrate recruitment of a new cue, object shape, for the appearance of rotating 3D objects. The cue was established by interleaving ambiguous and disambiguated instances of two shapes, cubes and spheres, at the same retinal location. Disambiguated cubes and spheres rotated in opposite directions. A significant bias was consequently introduced in the resolution of ambiguity, whereby the proportions of ambiguous shapes perceived as rotating clockwise differed, in the direction predicted by their disambiguated counterparts. This finding suggests that training led the visual system to distinguish between the two shapes. The association of rotation direction and shape was only achieved when monocular depth cues were used to depict rotation in depth; shapes disambiguated by binocular disparity did not lead to recruitment of the shape cue. We speculate that this difference may be the consequence of a difference in the neural pathways by which the disambiguating cues act. This new instance of the cue recruitment effect opens possibilities for further generalization of the phenomenon.

Disambiguation of Necker cube rotation by monocular and binocular depth cues: Relative effectiveness for establishing long-term bias

The apparent direction of rotation of perceptually bistable wire-frame (Necker) cubes can be conditioned to depend on retinal location by interleaving their presentation with cubes that are disambiguated by depth cues (Haijiang, Saunders, Stone, & Backus, 2006; Harrison & Backus, 2010a). The long-term nature of the learned bias is demonstrated by resistance to counter-conditioning on a consecutive day. In previous work, either binocular disparity and occlusion, or a combination of monocular depth cues that included occlusion, internal occlusion, haze, and depth-from-shading, were used to control the rotation direction of disambiguated cubes. Here, we test the relative effectiveness of these two sets of depth cues in establishing the retinal location bias. Both cue sets were highly effective in establishing a perceptual bias on Day 1 as measured by the perceived rotation direction of ambiguous cubes. The effect of counter-conditioning on Day 2, on perceptual outcome for ambiguous cubes, was independent of whether the cue set was the same or different as Day 1. This invariance suggests that a common neural population instantiates the bias for rotation direction, regardless of the cue set used. However, in a further experiment where only disambiguated cubes were presented on Day 1, perceptual outcome of ambiguous cubes during Day 2 counter-conditioning showed that the monocular-only cue set was in fact more effective than disparity-plus-occlusion for causing long-term learning of the bias. These results can be reconciled if the conditioning effect of Day 1 ambiguous trials in the first experiment is taken into account (Harrison & Backus, 2010b). We suggest that monocular disambiguation leads to stronger bias either because it more strongly activates a single neural population that is necessary for perceiving rotation, or because ambiguous stimuli engage cortical areas that are also engaged by monocularly disambiguated stimuli but not by disparity-disambiguated stimuli.

Stereoscopic Offset Makes Objects Easier to Recognize

Binocular vision is obviously useful for depth perception, but it might also enhance other components of visual processing, such as image segmentation. We used naturalistic images to determine whether giving an object a stereoscopic offset of 15-120 arcmin of crossed disparity relative to its background would make the object easier to recognize in briefly presented (33-133 ms), temporally masked displays. Disparity had a beneficial effect across a wide range of disparities and display durations. Most of this benefit occurred whether or not the stereoscopic contour agreed with the object’s luminance contour. We attribute this benefit to an orienting of spatial attention that selected the object and its local background for enhanced 2D pattern processing. At longer display durations, contour agreement provided an additional benefit, and a separate experiment using random-dot stimuli confirmed that stereoscopic contours plausibly contributed to recognition at the longer display durations in our experiment. We conclude that in real-world situations binocular vision confers an advantage not only for depth perception, but also for recognizing objects from their luminance patterns and bounding contours.

A trained perceptual bias that lasts for weeks

Classical (Pavlovian) conditioning procedures can be used to bias the appearance of physical stimuli. Under natural conditions this form of perceptual learning could cause perception to become more accurate by changing prior belief to be in accord with what is statistically likely. However, for learning to be of functional significance, it must last until similar stimuli are encountered again. Here, we used the apparent rotation direction of a revolving wire frame (Necker) cube to test whether a learned perceptual bias is long lasting. Apparent rotation direction was trained to have a different bias at two different retinal locations by interleaving the presentation of ambiguous cubes with presentation of cubes that were disambiguated by disparity and occlusion cues. Four groups of eight subjects were subsequently tested either 1, 7, 14, or 28 days after initial training, respectively, using a counter-conditioning procedure. All four groups showed incomplete re-learning of the reversed contingency relationship during their second session. One group repeated the counter-conditioning and showed an increase in the reverse bias, showing that the first counter-conditioning session also had a long-lasting effect. The fact that the original learning was still evident four weeks after the initial training is consistent with the operation of a mechanism that ordinarily would improve the accuracy and efficiency of perception.

Stimulus characteristics affect assessment of pupil defects in amblyopia.

Purpose Amblyopes do not reliably show relative afferent pupillary defects with full-field stimulation, but amblyopia has cortical involvement; hence, stimuli that engage cortex may be able to reveal pupil defects in amblyopes. Methods Pupillary responses were acquired with a binocular infrared pupillometer (RAPDx, Konan Medical USA, Irvine, CA) from 15 amblyopic subjects (anisometropic and small-angle strabismic) and 10 age-matched control subjects. Stimuli were a full-field white flash (330 cd/m2) or a small (4 degrees) annulus at one of three contrast levels (0.3, 0.6, and 1.8) on a dim background (6.2 cd/m2). Stimulus duration was 100 milliseconds, and the interstimulus duration was 2000 milliseconds. Results In all four stimulus conditions, the difference in percent contraction amplitude for right versus left eye stimulation was more variable across amblyopes than across control subjects. Amblyopic eyes did not show a specific deficit for the full-field flash. However, the mid-contrast (0.6) annulus stimulus revealed a deficit in the amblyopic eye, whereas the size of the deficit did not correlate with the type or depth of the amblyopia. Conclusions Targets of appropriate pattern, brightness, and contrast that select for cortical contributions to the pupil response may be useful for eliciting pupil defects in amblyopic patients. Pupil analysis in this population could prove useful for diagnostic or prognostic value, for example, to determine which amblyopes will respond best to treatment.