Thomas S. A. Wallis

Staying focused: A functional account of perceptual suppression during binocular rivalry

Presenting different images to either eye can induce perceptual switching, with alternating disappearances of each image--a phenomenon called binocular rivalry. We believe that disappearances during binocular rivalry can be driven by a process that facilitates visibility near the point of fixation. As the point of fixation is tied neither to a particular stimulus nor to a specific eye, indifference to both would be an essential characteristic for the process we envisage. Many factors that influence disappearances during binocular rivalry scale with distance in depth from fixation. Of these, here we use blur. We break the links between this cue and both eye of origin and stimulus type. We find that perceptual dominance can track a better focused image as it is swapped between the eyes and that perceptual switches can be driven by alternating the focus of images fixed in each eye. This implies that, as a determinant of suppression selectivity, blur is functionally independent from both eye of origin and stimulus type. Our data and theoretical account suggest that binocular rivalry is not an irrelevant laboratory curiosity but, rather, that it is a product of a functional adaptation that promotes visibility in cluttered environments.

Visual Crowding Is Correlated with Awareness

Crowding by nearby features causes identification failures in the peripheral visual field. However, prominent visual features can sometimes fail to reach awareness, causing scenes to be incorrectly interpreted. Here we examine whether awareness of the flanking features is necessary for crowding to occur. Flankers that were physically present were rendered perceptually absent with adaptation-induced blindness. In a letter identification task, targets were presented unflanked or with up to four flanker letters. On each trial, observers reported both the number of letters they perceived and the identity of a target letter. This paradigm allowed trial-by-trial assessment of awareness and crowding and ensured that both targets and flankers were attended. Target-letter identification performance was correlated with the number of flanking letters that were perceived on a given trial, regardless of the number that were physically present. Our data demonstrate that crowding can be released when flanking elements at attended locations are suppressed from visual awareness.

Information-theoretic model comparison unifies saliency metrics

Significance Where do people look in images? Predicting eye movements from images is an active field of study, with more than 50 quantitative prediction models competing to explain scene viewing behavior. Yet the rules for this competition are unclear. Using a principled metric for model comparison (information gain), we quantify progress in the field and show how formulating the models probabilistically resolves discrepancies in other metrics. We have also developed model assessment tools to reveal where models fail on the database, image, and pixel levels. These tools will facilitate future advances in saliency modeling and are made freely available in an open source software framework (www.bethgelab.org/code/pysaliency). Learning the properties of an image associated with human gaze placement is important both for understanding how biological systems explore the environment and for computer vision applications. There is a large literature on quantitative eye movement models that seeks to predict fixations from images (sometimes termed “saliency” prediction). A major problem known to the field is that existing model comparison metrics give inconsistent results, causing confusion. We argue that the primary reason for these inconsistencies is because different metrics and models use different definitions of what a “saliency map” entails. For example, some metrics expect a model to account for image-independent central fixation bias whereas others will penalize a model that does. Here we bring saliency evaluation into the domain of information by framing fixation prediction models probabilistically and calculating information gain. We jointly optimize the scale, the center bias, and spatial blurring of all models within this framework. Evaluating existing metrics on these rephrased models produces almost perfect agreement in model rankings across the metrics. Model performance is separated from center bias and spatial blurring, avoiding the confounding of these factors in model comparison. We additionally provide a method to show where and how models fail to capture information in the fixations on the pixel level. These methods are readily extended to spatiotemporal models of fixation scanpaths, and we provide a software package to facilitate their use.

Motion-induced blindness is not tuned to retinal speed

Motion-induced blindness is a visual phenomenon in which a moving pattern can cause superimposed static targets that remain physically present to intermittently disappear from awareness. To date, there has been little systematic investigation of the type of motion that induces the most robust perceptual disappearances. To address this issue, we investigated the temporal frequency and stimulus speed sensitivity of this phenomenon in two experiments. In the first, we used radial gratings and waveform modulation to decouple spatiotemporal frequency and retinal speed characteristics. The results suggested that motion-induced disappearances are tuned to temporal frequency, but not to stimulus speed. In the second, we showed that luminance flicker-induced disappearances were tuned to the same temporal frequency as motion-induced disappearances. In combination, these data suggest that motion-induced blindness does not depend on retinal stimulus speed. Rather, it seems to be broadly tuned for moderate rates of temporal modulation. This observation is reminiscent of other instances where motion and spatial coding interact to modulate visibility.

Testing models of peripheral encoding using metamerism in an oddity paradigm.

Most of the visual field is peripheral, and the periphery encodes visual input with less fidelity compared to the fovea. What information is encoded, and what is lost in the visual periphery? A systematic way to answer this question is to determine how sensitive the visual system is to different kinds of lossy image changes compared to the unmodified natural scene. If modified images are indiscriminable from the original scene, then the information discarded by the modification is not important for perception under the experimental conditions used. We measured the detectability of modifications of natural image structure using a temporal three-alternative oddity task, in which observers compared modified images to original natural scenes. We consider two lossy image transformations, Gaussian blur and Portilla and Simoncelli texture synthesis. Although our paradigm demonstrates metamerism (physically different images that appear the same) under some conditions, in general we find that humans can be capable of impressive sensitivity to deviations from natural appearance. The representations we examine here do not preserve all the information necessary to match the appearance of natural scenes in the periphery.

Perceived Size and Spatial Coding

Images of the same physical dimensions on the retina can appear to represent different-sized objects. One reason for this is that the human visual system can take viewing distance into account when judging apparent size. Sequentially presented images can also prompt spatial coding interactions. Here we show, using a spatial coding phenomenon (the tilt aftereffect) in tandem with viewing distance cues, that the tuning of such interactions is not simply determined by the physical dimensions of retinal input. Rather, we find that they are contingent on apparent size. Our data therefore reveal that spatial coding interactions in human vision are modulated by processes involved in the determination of apparent size.

Spatiotemporal Rivalry: A Perceptual Conflict Involving Illusory Moving and Static Forms

In human vision, mechanisms specialized for encoding static form can signal the presence of blurred forms trailing behind moving objects. People are typically unaware of these motion-blur signals because other mechanisms signal sharply defined moving forms. When active, these mechanisms can suppress awareness of motion blur. Thus, although discrepant form signals can be produced, human vision usually settles on a single coherent perceptual outcome. Here we report a dramatic exception. We found that, in some circumstances, static motion-blur form signals and moving-form signals can engage in a dynamic competition for perceptual dominance. We refer to the phenomenon as spatiotemporal rivalry (STR). Our data confirm that moving- and static-form mechanisms can generate independent signals, each of which can intermittently dominate perception. STR could therefore be exploited to investigate how these mechanisms contribute to determining the content of visual awareness.

Understanding Low- and High-Level Contributions to Fixation Prediction

Understanding where people look in images is an important problem in computer vision. Despite significant research, it remains unclear to what extent human fixations can be predicted by low-level (contrast) compared to highlevel (presence of objects) image features. Here we address this problem by introducing two novel models that use different feature spaces but the same readout architecture. The first model predicts human fixations based on deep neural network features trained on object recognition. This model sets a new state-of-the art in fixation prediction by achieving top performance in area under the curve metrics on the MIT300 hold-out benchmark (AUC = 88%, sAUC = 77%, NSS = 2.34). The second model uses purely low-level (isotropic contrast) features. This model achieves better performance than all models not using features pretrained on object recognition, making it a strong baseline to assess the utility of high-level features. We then evaluate and visualize which fixations are better explained by lowlevel compared to high-level image features. Surprisingly we find that a substantial proportion of fixations are better explained by the simple low-level model than the stateof- the-art model. Comparing different features within the same powerful readout architecture allows us to better understand the relevance of low- versus high-level features in predicting fixation locations, while simultaneously achieving state-of-the-art saliency prediction.

Characterization of field loss based on microperimetry is predictive of face recognition difficulties.

PURPOSE To determine how visual field loss as assessed by microperimetry is correlated with deficits in face recognition. METHODS Twelve patients (age range, 26-70 years) with impaired visual sensitivity in the central visual field caused by a variety of pathologies and 12 normally sighted controls (control subject [CS] group; age range, 20-68 years) performed a face recognition task for blurred and unblurred faces. For patients, we assessed central visual field loss using microperimetry, fixation stability, Pelli-Robson contrast sensitivity, and letter acuity. RESULTS Patients were divided into two groups by microperimetry: a low vision (LV) group (n = 8) had impaired sensitivity at the anatomical fovea and/or poor fixation stability, whereas a low vision that excluded the fovea (LV:F) group (n = 4) was characterized by at least some residual foveal sensitivity but insensitivity in other retinal regions. The LV group performed worse than the other groups at all blur levels, whereas the performance of the LV:F group was not credibly different from that of the CS group. The performance of the CS and LV:F groups deteriorated as blur increased, whereas the LV group showed consistently poor performance regardless of blur. Visual acuity and fixation stability were correlated with face recognition performance. CONCLUSIONS Persons diagnosed as having disease affecting the central visual field can recognize faces as well as persons with no visual disease provided that they have residual sensitivity in the anatomical fovea and show stable fixation patterns. Performance in this task is limited by the upper resolution of nonfoveal vision or image blur, whichever is worse.

Sensitivity to gaze-contingent contrast increments in naturalistic movies: An exploratory report and model comparison.

Sensitivity to luminance contrast is a prerequisite for all but the simplest visual systems. To examine contrast increment detection performance in a way that approximates the natural environmental input of the human visual system, we presented contrast increments gaze-contingently within naturalistic video freely viewed by observers. A band-limited contrast increment was applied to a local region of the video relative to the observers current gaze point, and the observer made a forced-choice response to the location of the target (≈25,000 trials across five observers). We present exploratory analyses showing that performance improved as a function of the magnitude of the increment and depended on the direction of eye movements relative to the target location, the timing of eye movements relative to target presentation, and the spatiotemporal image structure at the target location. Contrast discrimination performance can be modeled by assuming that the underlying contrast response is an accelerating nonlinearity (arising from a nonlinear transducer or gain control). We implemented one such model and examined the posterior over model parameters, estimated using Markov-chain Monte Carlo methods. The parameters were poorly constrained by our data; parameters constrained using strong priors taken from previous research showed poor cross-validated prediction performance. Atheoretical logistic regression models were better constrained and provided similar prediction performance to the nonlinear transducer model. Finally, we explored the properties of an extended logistic regression that incorporates both eye movement and image content features. Models of contrast transduction may be better constrained by incorporating data from both artificial and natural contrast perception settings.