Jan Tünnermann

Region-Based Artificial Visual Attention in Space and Time

Mobile robots have to deal with an enormous amount of visual data containing static and dynamic stimuli. Depending on the task, only small portions of a scene are relevant. Artificial attention systems filter information at early stages. Among the various methods proposed to implement such systems, the region-based approach has proven to be robust and especially suited for integrating top-down influences. This concept was recently transferred to the spatiotemporal domain to obtain motion saliency. A full-featured integration of the spatial and spatiotemporal systems is presented here. We propose a biologically inspired two-stream system, which allows to use different spatial and temporal resolutions and to pick off spatiotemporal saliency at early stages. We compare the output to classic models and demonstrate the flexibility of the integrated approach in different experiments. These include online processing of continuous input, a task similar to thumbnail extraction and a top-down task of selecting specific moving and non-moving objects.

Parallel k-Means Image Segmentation Using Sort, Scan and Connected Components on a GPU

Image segmentation is required to run fast and without supervision to speed up subsequent processes such as object recognition or other high level tasks. General purpose computing on the GPU is a powerful tool to perform efficient image processing and has been applied to the image segmentation problem. However, state-of-the-art approaches still perform parts of the computations on the CPU requiring costly data exchange with the main memory. In this paper we suggest a fully unsupervised color image segmentation that runs completely on the GPU including the calculation of region features. We compare our results to a popular CPU-based and a recent GPU-based method and report a computation time advantage.

Fast and Conspicuous? Quantifying Salience With the Theory of Visual Attention.

Particular differences between an object and its surrounding cause salience, guide attention, and improve performance in various tasks. While much research has been dedicated to identifying which feature dimensions contribute to salience, much less regard has been paid to the quantitative strength of the salience caused by feature differences. Only a few studies systematically related salience effects to a common salience measure, and they are partly outdated in the light of new findings on the time course of salience effects. We propose Bundesen’s Theory of Visual Attention (TVA) as a theoretical basis for measuring salience and introduce an empirical and modeling approach to link this theory to data retrieved from temporal-order judgments. With this procedure, TVA becomes applicable to a broad range of salience-related stimulus material. Three experiments with orientation pop-out displays demonstrate the feasibility of the method. A 4th experiment substantiates its applicability to the luminance dimension.

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments

This protocol describes how to conduct temporal-order experiments to measure visual processing speed and the attentional resource distribution. The proposed method is based on a new and synergistic combination of three components: the temporal-order judgments (TOJ) paradigm, Bundesens Theory of Visual Attention (TVA), and a hierarchical Bayesian estimation framework. The method provides readily interpretable parameters, which are supported by the theoretical and neurophysiological underpinnings of TVA. Using TOJs, TVA-based estimates can be obtained for a broad range of stimuli, whereas traditional paradigms used with TVA are mainly limited to letters and digits. Finally, the meaningful parameters of the proposed model allow for the establishment of a hierarchical Bayesian model. Such a statistical model allows assessing results in one coherent analysis both on the subject and the group level. To demonstrate the feasibility and versatility of this new approach, three experiments are reported with attention manipulations in synthetic pop-out displays, natural images, and a cued letter-report paradigm.

Measuring and modeling salience with the theory of visual attention

For almost three decades, the theory of visual attention (TVA) has been successful in mathematically describing and explaining a wide variety of phenomena in visual selection and recognition with high quantitative precision. Interestingly, the influence of feature contrast on attention has been included in TVA only recently, although it has been extensively studied outside the TVA framework. The present approach further develops this extension of TVA’s scope by measuring and modeling salience. An empirical measure of salience is achieved by linking different (orientation and luminance) contrasts to a TVA parameter. In the modeling part, the function relating feature contrasts to salience is described mathematically and tested against alternatives by Bayesian model comparison. This model comparison reveals that the power function is an appropriate model of salience growth in the dimensions of orientation and luminance contrast. Furthermore, if contrasts from the two dimensions are combined, salience adds up additively.

Peripheral Visual Cues: Their Fate in Processing and Effects on Attention and Temporal-Order Perception

Peripheral visual cues lead to large shifts in psychometric distributions of temporal-order judgments. In one view, such shifts are attributed to attention speeding up processing of the cued stimulus, so-called prior entry. However, sometimes these shifts are so large that it is unlikely that they are caused by attention alone. Here we tested the prevalent alternative explanation that the cue is sometimes confused with the target on a perceptual level, bolstering the shift of the psychometric function. We applied a novel model of cued temporal-order judgments, derived from Bundesens Theory of Visual Attention. We found that cue–target confusions indeed contribute to shifting psychometric functions. However, cue-induced changes in the processing rates of the target stimuli play an important role, too. At smaller cueing intervals, the cue increased the processing speed of the target. At larger intervals, inhibition of return was predominant. Earlier studies of cued TOJs were insensitive to these effects because in psychometric distributions they are concealed by the conjoint effects of cue–target confusions and processing rate changes.

Affordance Estimation Enhances Artificial Visual Attention: Evidence from a Change-Blindness Study

Abstract Visual attention models are typically based on the concept of saliency, a conspicuity measure which considers features such as color, intensity or orientation. Much current research aims at modeling top-down interactions, which highly influence human attentional behavior. Typically, these are in the form of targets to be searched for or general characteristics (gist) of a scene. In humans, it has been shown that objects that afford actions, for example, graspable objects, strongly attract attention. Here, we integrate an artificial attention framework with a measure of affordances estimated from a sparse 3D scene representation. This work contributes further evidence for human attention being biased toward objects of high affordance, which for the first time is measured in an objective way. Furthermore, it demonstrates that artificial attention systems benefit from affordance estimation for predicting human attention. For technical systems, considering affordances provides mid-level influences that are not too specific or too general, but can guide attention toward potential action targets with respect to a system’s physical capabilities. Finally, the change-detection task we employ for model comparison constitutes a new method to evaluate artificial systems with respect to early human vision in natural scene perception.

Integrating Object Affordances with Artificial Visual Attention

Affordances, e.g., grasping possibilities, play a role in the guidance of human attention. We report experiments on the integration of affordance estimation with artificial visual attention in a prototypical model. Furthermore, Growing Neural Gas is discussed as a potential framework for future attention models that deeply integrate affordance, saliency and further attentional mechanisms.

Poking Left To Be Right? A Model-Based Analysis of Temporal Order Judged by Mice

The theory of visual attention (TVTVA) provides a formal framework for the assessment of visual attention and related processes. Its center is a mathematical model of visual encoding processes and discretely defined components of attention. Building on this model, TVTVA offers quantitative and process-related explanations for a variety of phenomena in the domain of visual attention. Because the theory relies on very general assumptions which might hold true for other domains of sensory processing, we tested its possible explanatory value for tactile processing in mice. Reanalyzing published data of temporal-order judgments by mice, we show how a TVTVA-based analysis identifies the processes which drive observable behavior and that it comes to conclusions quite different from those of conventional analyses of temporal-order judgments. According to this analysis, despite the same overall capacity dedicated to the task, some mice assume attentional biases toward one side, possibly to optimize their overall performance. We suggest that TVTVAs concepts provide a powerful point of vantage to find explanations for observable behavior where conventional analysis easily leads to dead ends.

Early top-down influences in control of attention: evidence from the attentional blink

The relevance of top-down information in the deployment of attention has more and more been emphasized in cognitive psychology. We present recent findings about the dynamic of these processes and also demonstrate that task relevance can be adjusted rapidly by incoming bottom-up information. This adjustment substantially increases performance in a subsequent task. Implications for artificial visual models are discussed.