Albert L. Rothenstein

Attention links sensing to recognition

This paper presents arguments that explicit strategies for visual attentional selection are important for cognitive vision systems, and shows that a number of proposals currently exist for exactly how parts of this goal may be accomplished. A comprehensive survey of approaches to computational attention is given. A key characteristic of virtually all the models surveyed here is that they receive significant inspiration from the neurobiology and psychophysics of human and primate vision. This, although not necessarily a key component of mainstream computer vision, seems very appropriate for cognitive vision systems given a definition of the topic that always includes the goal of human-like visual performance. A particular model, the Selective Tuning model, is overviewed in some detail. The growing neurobiological and psychophysical evidence for its biological plausibility is cited highlighting the fact that it has more biological support than other models; it is further claimed that it may form an appropriate starting point for the difficult task of integrating attention into cognitive vision systems.

Attentional Modulation and Selection – An Integrated Approach

Various models of the neural mechanisms of attentional modulation in the visual cortex have been proposed. In general, these models assume that an ‘attention’ parameter is provided separately. Its value as well as the selection of neuron(s) to which it applies are assumed, but its source and the selection mechanism are unspecified. Here we show how the Selective Tuning model of visual attention can account for the modulation of the firing rate at the single neuron level, and for the temporal pattern of attentional modulations in the visual cortex, in a self-contained formulation that simultaneously determines the stimulus elements to be attended while modulating the relevant neural processes.

TarzaNN: a general purpose neural network simulator for visual attention modeling

A number of computational models of visual attention exist, but making comparisons is difficult due to the incompatible implementations and levels at which the simulations are conducted. To address this issue, we have developed a general-purpose neural network simulator that allows all of these models to be implemented in a unified framework. The simulator allows for the distributed execution of models, in a heterogeneous environment. Graphical tools are provided for the development of models by non-programmers and a common model description format facilitates the exchange of models. In this paper we will present the design of the simulator and results that demonstrate its generality.

VISUAL FEATURE BINDING WITHIN THE SELECTIVE TUNING ATTENTION FRAMEWORK

We present a biologically plausible computational model for solving the visual feature binding problem, based on recent results regarding the time course and processing sequence in the primate visual system. The feature binding problem appears due to the distributed nature of visual processing in the primate brain, and the gradual loss of spatial information along the processing hierarchy. This paper puts forward the proposal that by using multiple passes of the visual processing hierarchy, both bottom-up and top-down, and using task information to tune the processing prior to each pass, we can explain the different recognition behaviors that primate vision exhibits. To accomplish this, four different kinds of binding processes are introduced and are tied directly to specific recognition tasks and their time course. The model relies on the reentrant connections so ubiquitous in the primate brain to recover spatial information, and thus allow features represented in different parts of the brain to be integrated in a unitary conscious percept. We show how different tasks and stimuli have different binding requirements, and present a unified framework within the Selective Tuning model of visual attention.

Selective tuning: feature binding through selective attention

We present a biologically plausible computational model for solving the visual binding problem. The binding problem appears due to the distributed nature of visual processing in the primate brain, and the gradual loss of spatial information along the processing hierarchy. The model relies on the reentrant connections so ubiquitous in the primate brain to recover spatial information, and thus allow features represented in different parts of the brain to be integrated in a unitary conscious percept. We demonstrate the ability of the Selective Tuning (ST) model of visual attention [1] to recover spatial information, and based on this propose a general solution to the binding problem. The solution is demonstrated on two classic problems: recovery of form from motion and binding of shape and color. We also demonstrate how the method is able to handle difficult situations such as occlusions and transparency. The model is discussed in relation to recent results regarding the time course and processing sequence for form-from-motion in the primate visual system.

Second-Order (non-fourier) attention-based face detection

We present an attention-based face detection and localization system. The system is biologically motivated, combining face detection based on second-order circular patterns with the localization capabilities of the Selective Tuning (ST) model of visual attention [1]. One of the characteristics of this system is that the face detectors are relatively insensitive to the scale and location of the face, and thus additional processing needs to be performed to localize the face for recognition. We extend STs ability to recover spatial information to this object recognition system, and show how this can be used to precisely localize faces in images. The system presented in this paper exhibits temporal characteristics that are qualitatively similar to those of the primate visual system in that detection and categorization is performed early in the processing cycle, while detailed information needed for recognition is only available after additional processing, consistent with experimental data and with certain theories of visual object recognition [2].

The Role of Attention in Shaping Visual Perceptual Processes

It has been known now for over 20 years that an optimal solution to a basic vision problem such as visual search, which is robust enough to apply to any possible image or target, is unattainable because the problem of visual search is provably intractable (“Tsotsos, The complexity of perceptual search tasks, Proceedings of the International Joint Conference on Artificial Intelligence, 1989,” “Rensink, A new proof of the NP-completeness of visual match, Technical Report 89–22, University of British Columbia, 1989”). That the brain seems to solve it in an apparently effortless manner then poses a mystery. Either the brain is performing in a manner that cannot be captured computationally, or it is not solving that same generic visual search problem. The first option has been shown to not be the case (“Tsotsos and Bruce, Scholarpedia, 3(12), 6545, 2008”). As a result, this chapter will focus on the second possibility. There are two elements required to deal with this. The first is to show how the nature of the problem solved by the brain is fundamentally different from the generic one, and second to show how the brain might deal with those differences. The result is a biologically plausible and computationally well-founded account of how attentional mechanisms dynamically shape perceptual processes to achieve this seemingly effortless capacity that humans – and perhaps most seeing animals – possess.

Modeling the Dynamics of Feature Binding During Object-Selective Attention

We present a biologically plausible computational model for solving the visual feature binding problem. The binding problem appears to be due to the distributed nature of visual processing in the primate brain, and the gradual loss of spatial information along the processing hierarchy. The model relies on the reentrant connections so ubiquitous in the primate brain to recover spatial information, and thus allows features represented in different parts of the brain to be integrated in a unitary conscious percept. We demonstrate the ability of the Selective Tuning model of visual attention [1] to recover spatial information, and based on this we propose a general solution to the feature binding problem. The solution is used to simulate the results of a recent neurophysiology study on the binding of motion and color. The example demonstrates how the method is able to handle the difficult case of transparency.

Task and timing in visual processing

The study of visual perception abounds with examples of surprising results, and perhaps none of these has generated more controversy than the speed of object recognition. Some complex objects can be recognized with amazing speed even while attention is engaged on a different task. Some simple objects need lengthy attentional scrutiny, and performance breaks down in dual-task experiments [1]. These results are fundamental to our understanding of the visual cortex, as they clearly show the interplay of the representation of information in the brain, attentional mechanisms, binding and consciousness. We argue that the lack of a common terminology is a significant contributor to this controversy, and define several different levels of tasks as: Detection – is a particular item present in the stimulus, yes or no?; Localization – detection plus accurate location; Recognition – localization plus detailed description of stimulus; Understanding – recognition plus role of stimulus in the context of the scene. It is clear from performance results that detection is not possible for all stimuli, and the difference must be in the internal representation of the different stimuli. For detection to be possible, the fast, feed-forward activation of a neuron (or pool of neurons) must represent the detected stimulus, which is consistent with the experimental finding that only highly over-learned and biologically relevant stimuli or broad stimulus categories can be detected. In detection tasks localization is poor or absent [2], so location needs to be recovered based on this initial representation. Given that detailed location and extent information is only available in the early processing areas, this must be accomplished by the ubiquitous feedback connections in the visual cortex. Once the location of a stimulus has been recovered and distracters inhibited, one or more subsequent feed-forward passes through the system can create a detailed representation of the selected stimulus. Here we present a computational demonstration of how attention forms the glue between the sparse, fast, and parallel initial representation that supports object detection and the slow, serial, and detailed representations needed for full recognition. The Selective Tuning (ST) model of (object based) visual attention [3] can be used to recover the spatial location and extent of the visual information that has contributed to a categorical decision. This allows for the selective detailed processing of this information at the expense of other stimuli present in the image. The feedback and selective processing create the detailed population code corresponding to the attended stimulus. We suggest and demonstrate a possible binding mechanism by which this is accomplished in the context of ST, and show how this solution can account for existing experimental results.