Neil D. B. Bruce

Saliency, attention and visual search: an information theoretic approach

A proposal for saliency computation within the visual cortex is put forth based on the premise that localized saliency computation serves to maximize information sampled from ones environment. The model is built entirely on computational constraints but nevertheless results in an architecture with cells and connectivity reminiscent of that appearing in the visual cortex. It is demonstrated that a variety of visual search behaviors appear as emergent properties of the model and therefore basic principles of coding and information transmission. Experimental results demonstrate greater efficacy in predicting fixation patterns across two different data sets as compared with competing models.

Attention in Cognitive Systems

Attention in Scene Exploration.- On the Optimality of Spatial Attention for Object Detection.- Decoding What People See from Where They Look: Predicting Visual Stimuli from Scanpaths.- A Novel Hierarchical Framework for Object-Based Visual Attention.- Where Do We Grasp Objects? - An Experimental Verification of the Selective Attention for Action Model (SAAM).- Contextual Cueing and Saliency.- Integrating Visual Context and Object Detection within a Probabilistic Framework.- The Time Course of Attentional Guidance in Contextual Cueing.- Conspicuity and Congruity in Change Detection.- Spatiotemporal Saliency.- Spatiotemporal Saliency: Towards a Hierarchical Representation of Visual Saliency.- Motion Saliency Maps from Spatiotemporal Filtering.- Attentional Networks.- Model Based Analysis of fMRI-Data: Applying the sSoTS Framework to the Neural Basic of Preview Search.- Modelling the Efficiencies and Interactions of Attentional Networks.- The JAMF Attention Modelling Framework.- Attentional Modeling.- Modeling Attention and Perceptual Grouping to Salient Objects.- Attention Mechanisms in the CHREST Cognitive Architecture.- Modeling the Interactions of Bottom-Up and Top-Down Guidance in Visual Attention.- Relative Influence of Bottom-Up and Top-Down Attention.- Towards Standardization of Evaluation Metrics and Methods for Visual Attention Models.- Comparing Learning Attention Control in Perceptual and Decision Space.- Automated Visual Attention Manipulation.

Features that draw visual attention: an information theoretic perspective

A novel image operator is proposed for the purpose of predicting the focus of visual attention in arbitrary natural scenes based on local statistics. The proposed method is based on the hypothetical premise that attention proceeds by way of sampling a scene in a manner that maximizes the information acquired from the scene. A tractable means of computing the joint likelihood of local statistics in a low-dimensional space is presented and shown to have a close relationship to the representation of retinal image stimulus existing in the primary visual cortex of primates. The proposed image operator is validated through comparison with existing features implicated in the focus of attention in their relative correlation to experimental eye tracking data.

An attentional framework for stereo vision

The necessity and utility of visual attention are discussed in the context of stereo vision in machines and primates. Specific problems that arise in this domain including binocular rivalry, and the deployment of attention in three-dimensional space are considered. Necessary conditions are outlined for achieving appropriate attentional behaviour in both the aforementioned domains. In this light, we outline classes of existing computational models of attention and discuss their applicability for realizing binocular attention. Finally, a stereo attention framework is presented by considering the tenets of an existing attentional architecture that extends naturally to the binocular domain, in conjunction with the connectivity of units involved in achieving stereo vision.

On computational modeling of visual saliency: Examining what's right, and what's left.

In the past decade, a large number of computational models of visual saliency have been proposed. Recently a number of comprehensive benchmark studies have been presented, with the goal of assessing the performance landscape of saliency models under varying conditions. This has been accomplished by considering fixation data, annotated image regions, and stimulus patterns inspired by psychophysics. In this paper, we present a high-level examination of challenges in computational modeling of visual saliency, with a heavy emphasis on human vision and neural computation. This includes careful assessment of different metrics for performance of visual saliency models, and identification of remaining difficulties in assessing model performance. We also consider the importance of a number of issues relevant to all saliency models including scale-space, the impact of border effects, and spatial or central bias. Additionally, we consider the biological plausibility of models in stepping away from exemplar input patterns towards a set of more general theoretical principles consistent with behavioral experiments. As a whole, this presentation establishes important obstacles that remain in visual saliency modeling, in addition to identifying a number of important avenues for further investigation.

An Attentional Mechanism for Selecting Appropriate Actions Afforded by Graspable Objects

An object may afford a number of different actions. In this article, we show that an attentional mechanism inhibits competing motor programs that could elicit erroneous actions. Participants made a speeded key press to categorize the second of two successively presented door handles that were rotated at varying orientations relative to one another. Their responding hand was compatible or incompatible with the graspable part of the door handles (rightward or leftward facing). Compatible responses were faster than incompatible responses if the two handles shared an identical orientation, but they were slower if the two handles were aligned at slightly dissimilar orientations. Such suppressive surround effects are hallmarks of attentional processing in the visual domain, but they have never been observed behaviorally in the motor domain. This finding delineates a common mechanism involved in two of the most important functions of the brain: processing sensory data and preparing actions based on that information.

Finding an emotional face in a crowd: Emotional and perceptual stimulus factors influence visual search efficiency

In this article, we examine how emotional and perceptual stimulus factors influence visual search efficiency. In an initial task, we run a visual search task, using a large number of target/distractor emotion combinations. In two subsequent tasks, we then assess measures of perceptual (rated and computational distances) and emotional (rated valence, arousal and potency) stimulus properties. In a series of regression analyses, we then explore the degree to which target salience (the size of target/distractor dissimilarities) on these emotional and perceptual measures predict the outcome on search efficiency measures (response times and accuracy) from the visual search task. The results show that both emotional and perceptual stimulus salience contribute to visual search efficiency. The results show that among the emotional measures, salience on arousal measures was more influential than valence salience. The importance of the arousal factor may be a contributing factor to contradictory history of results within this field.

Spatiotemporal Saliency: Towards a Hierarchical Representation of Visual Saliency

In prior work, we put forth a model of visual saliency motivated by information theoretic considerations [1]. In this effort we consider how this proposal extends to explain saliency in the spatiotemporal domain and further, propose a distributed representation for visual saliency comprised of localized hierarchical saliency computation. Evidence for the efficacy of the proposal in capturing aspects of human behavior is achieved via comparison with eye tracking data and a discussion of the role of neural coding in the determination of saliency suggests avenues for future research.

An Information Theoretic Model of Saliency and Visual Search

In this paper, a proposal which quantifies visual saliency based on an information theoretic definition is evaluated with respect to visual psychophysics paradigms. Analysis reveals that the proposal explains a broad range of results from classic visual search tasks, including many for which only specialized models have had success. As a whole, the results provide strong behavioral support for a model of visual saliency based on information, supplementing earlier work revealing the efficacy of the approach in predicting primate fixation data.

A Deeper Look at Saliency: Feature Contrast, Semantics, and Beyond

In this paper we consider the problem of visual saliency modeling, including both human gaze prediction and salient object segmentation. The overarching goal of the paper is to identify high level considerations relevant to deriving more sophisticated visual saliency models. A deep learning model based on fully convolutional networks (FCNs) is presented, which shows very favorable performance across a wide variety of benchmarks relative to existing proposals. We also demonstrate that the manner in which training data is selected, and ground truth treated is critical to resulting model behaviour. Recent efforts have explored the relationship between human gaze and salient objects, and we also examine this point further in the context of FCNs. Close examination of the proposed and alternative models serves as a vehicle for identifying problems important to developing more comprehensive models going forward.