Xosé R. Fdez-Vidal

Saliency from hierarchical adaptation through decorrelation and variance normalization

This paper presents a novel approach to visual saliency that relies on a contextually adapted representation produced through adaptive whitening of color and scale features. Unlike previous models, the proposal is grounded on the specific adaptation of the basis of low level features to the statistical structure of the image. Adaptation is achieved through decorrelation and contrast normalization in several steps in a hierarchical approach, in compliance with coarse features described in biological visual systems. Saliency is simply computed as the square of the vector norm in the resulting representation. The performance of the model is compared with several state-of-the-art approaches, in predicting human fixations using three different eye-tracking datasets. Referring this measure to the performance of human priority maps, the model proves to be the only one able to keep the same behavior through different datasets, showing free of biases. Moreover, it is able to predict a wide set of relevant psychophysical observations, to our knowledge, not reproduced together by any other model before.

On the relationship between optical variability, visual saliency, and eye fixations: A computational approach

A hierarchical definition of optical variability is proposed that links physical magnitudes to visual saliency and yields a more reductionist interpretation than previous approaches. This definition is shown to be grounded on the classical efficient coding hypothesis. Moreover, we propose that a major goal of contextual adaptation mechanisms is to ensure the invariance of the behavior that the contribution of an image point to optical variability elicits in the visual system. This hypothesis and the necessary assumptions are tested through the comparison with human fixations and state-of-the-art approaches to saliency in three open access eye-tracking datasets, including one devoted to images with faces, as well as in a novel experiment using hyperspectral representations of surface reflectance. The results on faces yield a significant reduction of the potential strength of semantic influences compared to previous works. The results on hyperspectral images support the assumptions to estimate optical variability. As well, the proposed approach explains quantitative results related to a visual illusion observed for images of corners, which does not involve eye movements.

Decorrelation and Distinctiveness Provide with Human-Like Saliency

In this work, we show the capability of a new model of saliency, of reproducing remarkable psychophysical results. The model presents low computational complexity compared to other models of the state of the art. It is based in biologically plausible mechanisms: the decorrelation and the distinctiveness of local responses. Decorrelation of scales is obtained from principal component analysis of multiscale low level features. Distinctiveness is measured through the Hotelling’s T2 statistic. The model is conceived to be used in a machine vision system, in which attention would contribute to enhance performance together with other visual functions. Experiments demonstrate the consistency with a wide variety of psychophysical phenomena, that are referenced in the visual attention modeling literature, with results that outperform other state of the art models.

Information theoretic measure for visual target distinctness

It is of great benefit to have advance knowledge of human visual target acquisition performance for targets or other relevant objects. However, search performance inherently shows a large variance and depends strongly on prior knowledge of the perceived scene. A typical search experiment therefore requires a large number of observers to obtain statistically reliable data. Moreover, measuring target acquisition performance in field situations is usually impractical and often very costly or even dangerous. The paper presents a method for characterizing information of a target relative to its background. The resultant computational measures are then applied to quantify the visual distinctness of targets in complex natural backgrounds from digital imagery. A generalization of the Kullback-Leibler joint information gain of various random variables is shown to correlate strongly with visual target distinctness as estimated by human observers. Bootstrap methods for assessing statistical accuracy were used to produce this inference.

The RGFF representational model: a system for the automatically learned partitioning of "visual patterns" in digital images

This paper describes a system for the automatically learned partitioning of visual patterns in 2D images, based on sophisticated band-pass filtering with fixed scale and orientation sensitivity. The visual patterns are defined as the features which have the highest degree of alignment in the statistical structure across different frequency bands. The analysis reorganizes the image according to an invariance constraint in statistical structure and consists of three stages: pre-attentive stage, integration stage, and learning stage. The first stage takes the input image and performs filtering with log-Gabor filters. Based on their responses, activated filters which are selectively sensitive to patterns in the image are short listed. In the integration stage, common grounds between several activated sensors are explored. The filtered responses are analyzed through a family of statistics. For any given two activated filters, a distance between them is derived via distances between their statistics. The third stage performs cluster partitioning for learning the subspace of log-Gabor filters needed to partition the image data. The clustering is based on a dissimilarity measure intended to highlight scale and orientation invariance of the responses. The technique is illustrated on real and simulated data sets. Finally, this paper presents a computational visual distinctness measure computed from the image representational model based on visual patterns. Experiments are performed to investigate its relation to distinctness as measured by human observers.

The selection of natural scales in 2D images using adaptive Gabor filtering

This paper analyzes how the natural scales of the shapes in 2D images can be extracted. Spatial information is analyzed by multiple units sensitive to both spatial and spatial-frequency variables. Scale estimates of the relevant shapes are constructed only from strongly responding detectors. The meaningful structures in the response of a detector (computed through 2D Gabor filtering) are, at their natural level of resolution, relatively sharp and have well-defined boundaries. A natural scale is so defined as a level producing local minimum of a function that returns the relative sharpness of the detector response filtered over a range of scales. In a second stage, to improve a first crude estimate of the local scale, the criterion is also rewritten to directly select scales at locations of significant features of each activated detector.

Using models of feature perception in distortion measure guidance

Abstract In this paper we present three error measures based on feature perception models, in which pixel errors are computed on locations at which humans might perceive features in the reference image. In the first part of this work, the three schemes of feature detection will be discussed and evaluated in terms of their performance for a simple visual signal-processing task. The first model is based on the use of local intensity gradients, the second based on the use of phase congruency in an image, and the third based on the use of local energy maxima for a few active sensors under a multichannel organization of the reference picture. In the second part of this paper, examples are provided of object detection and recognition applications that illustrate the ability of the induced error measures to predict the detectability of objects in natural backgrounds as well as their perceptual capabilities.

Computing visual target distinctness through selective filtering, statistical features, and visual patterns

This paper presents three computational visual distinctness measures, computed from image representational models based on selective filtering, statistical features, and visual patterns, respectively. They are applied to quantify the visual distinctness of targets in complex natural scenes. The measure that applies a simple decision rule to the distances between segregated visual patterns is shown (1) to predict human observer performance in search and detection tasks on complex natural imagery, and (2) to correlate strongly with visual target distinctness estimated by human observers.

Performance of the Kullback-Leibler information gain for predicting image fidelity

This paper presents a new method for characterizing information of a compressed image relative to the original one. We show how the Kullback-Leibler information gain is based on three basic postulates which are natural for image processing and thus desirable. As an example of the proposed measure, we analyze the effects of lossy compression on the identification of breast cancer microcalcifications. We also show the comparative results of the Kullback-Leibler information gain and various quantitative measures for predicting image fidelity in the sense of diagnostic usefulness.

A new image distortion measure based on a data-driven multisensor organization†

This paper describes a visual model that gives a perceptual distortion measure between an input image and that of reference based on a human-image representational model. We study an approach in which once a few active recognizers tuned to significant orientation and spatial-frequency components of the reference spectrum are obtained, any input image to be compared with the reference one is passed through an operator designated to compare its excitation levels given by the active recognizers, to the corresponding excitation levels for the reference image. Hence, the distortion between a pair of complex images is measured as the weighted sum of the distortion in each filter of a bank of strongly responding recognizers, each tuned to a certain 2D spatial-frequency data in the reference picture, with the weighting of each filter modulating its amplitude response.