Valentina I. Gusakova

Attention-Guided Recognition Based on “What” and “Where”: Representations: A Behavioral Model

ABSTRACT We describe the model of attention-guided visual perception and recognition previously published in Vision Research ( Rybak et al., 1998) . The model contains (1) a low-level subsystem that performs a fovealike transformation and detection of primary features (edges) and (2) a high-level subsystem that includes separated “what” (sensory memory) and “where” (motor memory) subsystems. In the model, image recognition occurs under top-down control of visual attention during the execution of a behavioral recognition program formed during the primary viewing of the image. The recognition program contains both programmed movements of an attention window (stored in the motor memory) and predicted image fragments (stored in the sensory memory) for each consecutive fixation. The model shows the ability to recognize complex images (e.g., faces) invariantly with respect to shift, rotation, and scale.We describe the model of attention-guided visual perception and recognition previously published in Vision Research ( Rybak et al., 1998). The model contains (1) a low-level subsystem that performs a fovealike transformation and detection of primary features (edges) and (2) a high-level subsystem that includes separated “what” (sensory memory) and “where” (motor memory) subsystems. In the model, image recognition occurs under top-down control of visual attention during the execution of a behavioral recognition program formed during the primary viewing of the image. The recognition program contains both programmed movements of an attention window (stored in the motor memory) and predicted image fragments (stored in the sensory memory) for each consecutive fixation. The model shows the ability to recognize complex images (e.g., faces) invariantly with respect to shift, rotation, and scale.

MARR: Active vision model

Earlier, the biologically plausible active vision, model for multiresolutional attentional representation and recognition (MARR) has been developed. The model is based on the scanpath theory of Noton and Stark and provides invariant recognition of gray-level images. In the present paper, the algorithm of automatic image viewing trajectory formation in the MARR model, the results of psychophysical experiments, and possible applications of the model are considered. Algorithm of automatic image viewing trajectory formation is based on imitation of the scanpath formed by operator. Several propositions about possible mechanisms for a consecutive selection of fixation points in human visual perception inspired by computer simulation results and known psychophysical data have been tested and confirmed in our psychophysical experiments. In particular, we have found that gaze switch may be directed (1) to a peripheral part of the vision field which contains an edge oriented orthogonally to the edge in the point of fixation, and (2) to a peripheral part of the vision field containing crossing edges. Our experimental results have been used to optimize automatic algorithm of image viewing in the MARR model. The modified model demonstrates an ability to recognize complex real world images invariantly with respect to scale, shift, rotation, illumination conditions, and, in part, to point of view and can be used to solve some robot vision tasks.

Model-based approach to study of mechanisms of complex image viewing

A model-based approach to study complex image viewing mechanisms and the first results of its implementation are presented. The choice of the most informative regions (MIRs) is performed according to results of psychophysical tests with high-accuracy tracking of eye movements. For three test images, the MIRs were determined as image regions with maximal density of gaze fixations for the all subjects (n = 9). Individual image viewing scanpaths (n= 49) were classified into three basic types (i.e. “viewing”, “object-consequent”, and “object-returned” scanpaths). Task-related and temporal dynamics of eye movement parameters for the same subjects have been found. Artificial image scanpaths similar to experimental have been obtained by means of gaze attraction function.

Attention-Guided Recognition Based on “What” and “Where”

ABSTRACT We describe the model of attention-guided visual perception and recognition previously published in Vision Research ( Rybak et al., 1998) . The model contains (1) a low-level subsystem that performs a fovealike transformation and detection of primary features (edges) and (2) a high-level subsystem that includes separated “what” (sensory memory) and “where” (motor memory) subsystems. In the model, image recognition occurs under top-down control of visual attention during the execution of a behavioral recognition program formed during the primary viewing of the image. The recognition program contains both programmed movements of an attention window (stored in the motor memory) and predicted image fragments (stored in the sensory memory) for each consecutive fixation. The model shows the ability to recognize complex images (e.g., faces) invariantly with respect to shift, rotation, and scale.We describe the model of attention-guided visual perception and recognition previously published in Vision Research ( Rybak et al., 1998). The model contains (1) a low-level subsystem that performs a fovealike transformation and detection of primary features (edges) and (2) a high-level subsystem that includes separated “what” (sensory memory) and “where” (motor memory) subsystems. In the model, image recognition occurs under top-down control of visual attention during the execution of a behavioral recognition program formed during the primary viewing of the image. The recognition program contains both programmed movements of an attention window (stored in the motor memory) and predicted image fragments (stored in the sensory memory) for each consecutive fixation. The model shows the ability to recognize complex images (e.g., faces) invariantly with respect to shift, rotation, and scale.

Formation of object representation for invariant image recognition using specific trajectories of viewing

A model of the foveal system for Multiresolutional Attentional Representation and Recognition (MARR) of grey-level images has been developed. The system performs the following procedures: analysis of an image at several resolution levels; parallel information processing in central (basic) and context (peripheral) regions of an attention window; estimation of similarity of both object fragments at each fixation point and viewing trajectories (object scanpaths) at the stages of memorizing (learning) and recognition.

Approach to invariant object recognition on grey-level images by exploiting neural network models

A model of a neural network system for object recognition in grey-level images that is invariant with respect to position, rotation, and scale is developed. The model is based on the theory of D. Noton and L. Stark and on the concept of smart sensing. A method for visual image invariant representation is proposed. The method allows transformation of primary features into invariant ones which can be used as input signals for a classical neural network classifier of the high-level structure of the recognizing system.© (1991) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A neural network system model for active perception and invariant recognition of grey-level images

A method for parallel-sequential processing of gray-level images and their representation which is invariant to position, rotation, and scale has been developed. The method is based on the idea that an image is memorized and recognized by way of consecutive fixations of moving eyes on the most informative image fragments. The method provides the invariant representation of the image in each fixation point and of spatial relations of features extracted in neighboring fixation points. A model of a neural network system for active visual perception and recognition of gray-level images has been developed based on this method. The experiments carried out with the model have shown that the system was able to recognize complex gray-level images in real time with invariance regarding position, rotation, and scale.<<ETX>>

Neural network modeling in studying neuron mechanisms of visual perception

An investigation including the development of realistic neural network models, computer experiments, and verification of their predictions in neurophysiological experiments made it possible to obtain a set of principally new facts on spatial and temporal dynamics of VC (visual cortex) neurons OS (orientation selectivity) and to develop a method of parallel-sequential processing of images. Attention was given to the temporal dynamics of the OS of the VC neurons, the spatial dynamics of the OS, and neural mechanisms of context description of visual object elements. Conformity of results of computer simulations and neurophysiological experiments supports the conceptual model of the iso-orientation domain described. The data obtained can be considered as an additional arguments in favor of the hypothesis that anisotropy of lateral inhibitory connections in the VC is one of the main mechanisms of formation of OS.<<ETX>>

CHAPTER 108 – Attention-Guided Recognition Based on “What” and “Where”: Representations: A Behavioral Model

ABSTRACT We describe the model of attention-guided visual perception and recognition previously published in Vision Research ( Rybak et al., 1998) . The model contains (1) a low-level subsystem that performs a fovealike transformation and detection of primary features (edges) and (2) a high-level subsystem that includes separated “what” (sensory memory) and “where” (motor memory) subsystems. In the model, image recognition occurs under top-down control of visual attention during the execution of a behavioral recognition program formed during the primary viewing of the image. The recognition program contains both programmed movements of an attention window (stored in the motor memory) and predicted image fragments (stored in the sensory memory) for each consecutive fixation. The model shows the ability to recognize complex images (e.g., faces) invariantly with respect to shift, rotation, and scale.We describe the model of attention-guided visual perception and recognition previously published in Vision Research ( Rybak et al., 1998). The model contains (1) a low-level subsystem that performs a fovealike transformation and detection of primary features (edges) and (2) a high-level subsystem that includes separated “what” (sensory memory) and “where” (motor memory) subsystems. In the model, image recognition occurs under top-down control of visual attention during the execution of a behavioral recognition program formed during the primary viewing of the image. The recognition program contains both programmed movements of an attention window (stored in the motor memory) and predicted image fragments (stored in the sensory memory) for each consecutive fixation. The model shows the ability to recognize complex images (e.g., faces) invariantly with respect to shift, rotation, and scale.

Modeling and investigation of some spatio-temporal aspects of visual information processing in the retinal neural network

A simplified retinal neural network (RNN) model has been considered. The main properties of this model are as follows: (1) primary transform of input raster simulates a decrease of resolution from the fovea to the retinal periphery; (2) the RNN consists of two layers, i.e., excitatory and inhibitory ones, each of them being formed by elements with identical properties excluding input transform; (3) each element of the excitatory layer is inhibited by the retinotopically corresponding element of the inhibitory layer; and (4) receptive field size and time constant of inhibitory neurons are more than those of excitatory ones. Two versions of the RNN differing in several aspects from each other were developed. In the first model the Gauss transform was used as a primary transform of the input raster. In addition, a wide range of the RNN and visual stimulus parameters was tested by computer simulation. The primary transform in the second model was performed by brightness averaging on neuron receptive fields. In the last case, qualitative behavior of the RNN was studied analytically. It was shown that neuron dynamics in response to moving stimuli and the preferable velocity of motion depended on neuron position in the RNN. In particular, foveal neurons were tuned to lower velocity as compared with peripheral ones.