Pierre Bayerl

Disambiguating Visual Motion Through Contextual Feedback Modulation

Motion of an extended boundary can be measured locally by neurons only orthogonal to its orientation (aperture problem) while this ambiguity is resolved for localized image features, such as corners or nonocclusion junctions. The integration of local motion signals sampled along the outline of a moving form reveals the object velocity. We propose a new model of V1-MT feedforward and feedback processing in which localized V1 motion signals are integrated along the feedforward path by model MT cells. Top-down feedback from MT cells in turn emphasizes model V1 motion activities of matching velocity by excitatory modulation and thus realizes an attentional gating mechanism. The model dynamics implement a guided filling-in process to disambiguate motion signals through biased on-center, off-surround competition. Our model makes predictions concerning the time course of cells in area MT and V1 and the disambiguation process of activity patterns in these areas and serves as a means to link physiological mechanisms with perceptual behavior. We further demonstrate that our model also successfully processes natural image sequences.

A Fast Biologically Inspired Algorithm for Recurrent Motion Estimation

We have previously developed a neurodynamical model of motion segregation in cortical visual area V1 and MT of the dorsal stream. The model explains how motion ambiguities caused by the motion aperture problem can be solved for coherently moving objects of arbitrary size by means of cortical mechanisms. The major bottleneck in the development of a reliable biologically inspired technical system with real-time motion analysis capabilities based on this neural model is the amount of memory necessary for the representation of neural activation in velocity space. We propose a sparse coding framework for neural motion activity patterns and suggest a means by which initial activities are detected efficiently. We realize neural mechanisms such as shunting inhibition and feedback modulation in the sparse framework to implement an efficient algorithmic version of our neural model of cortical motion segregation. We demonstrate that the algorithm behaves similarly to the original neural model and is able to extract image motion from real world image sequences. Our investigation transfers a neuroscience model of cortical motion computation to achieve technologically demanding constraints such as real-time performance and hardware implementation. In addition, the proposed biologically inspired algorithm provides a tool for modeling investigations to achieve acceptable simulation time

Disambiguating Visual Motion by Form-Motion Interaction--a Computational Model

The neural mechanisms underlying motion segregation and integration still remain unclear to a large extent. Local motion estimates often are ambiguous in the lack of form features, such as corners or junctions. Furthermore, even in the presence of such features, local motion estimates may be wrong if they were generated near occlusions or from transparent objects. Here, a neural model of visual motion processing is presented that involves early stages of the cortical dorsal and ventral pathways. We investigate the computational mechanisms of V1-MT feedforward and feedback processing in the perception of coherent shape motion. In particular, we demonstrate how modulatory MT-V1 feedback helps to stabilize localized feature signals at, e.g. corners, and to disambiguate initial flow estimates that signal ambiguous movement due to the aperture problem for single shapes. In cluttered environments with multiple moving objects partial occlusions may occur which, in turn, generate erroneous motion signals at points of overlapping form. Intrinsic-extrinsic region boundaries are indicated by local T-junctions of possibly any orientation and spatial configuration. Such junctions generate strong localized feature tracking signals that inject erroneous motion directions into the integration process. We describe a simple local mechanism of excitatory form-motion interaction that modifies spurious motion cues at T-junctions. In concert with local competitive-cooperative mechanisms of the motion pathway the motion signals are subsequently segregated into coherent representations of moving shapes. Computer simulations demonstrate the competency of the proposed neural model.

Detection of head pose and gaze direction for human-computer interaction

In this contribution we extend existing methods for head pose estimation and investigate the use of local image phase for gaze detection. Moreover we describe how a small database of face images with given ground truth for head pose and gaze direction was acquired. With this database we compare two different computational approaches for extracting the head pose. We demonstrate that a simple implementation of the proposed methods without extensive training sessions or calibration is sufficient to accurately detect the head pose for human-computer interaction. Furthermore, we propose how eye gaze can be extracted based on the outcome of local filter responses and the detected head pose. In all, we present a framework where different approaches are combined to a single system for extracting information about the attentional state of a person.

Sketching shiny surfaces: 3D shape extraction and depiction of specular surfaces

Many materials including water, plastic, and metal have specular surface characteristics. Specular reflections have commonly been considered a nuisance for the recovery of object shape. However, the way that reflections are distorted across the surface depends crucially on 3D curvature, suggesting that they could, in fact, be a useful source of information. Indeed, observers can have a vivid impression of, 3D shape when an object is perfectly mirrored (i.e., the image contains nothing but specular reflections). This leads to the question what are the underlying mechanisms of our visual system to extract this 3D shape information from a perfectly mirrored object. In this paper we propose a biologically motivated recurrent model for the extraction of visual features relevant for the perception of 3D shape information from images of mirrored objects. We qualitatively and quantitatively analyze the results of computational model simulations and show that bidirectional recurrent information processing leads to better results than pure feedforward processing. Furthermore, we utilize the model output to create a rough nonphotorealistic sketch representation of a mirrored object, which emphasizes image features that are mandatory for 3D shape perception (e.g., occluding contour and regions of high curvature). Moreover, this sketch illustrates that the model generates a representation of object features independent of the surrounding scene reflected in the mirrored object.

A Statistical Measure for Evaluating Regions-of-Interest Based Attention Algorithms

We present a new measure for evaluation of algorithms for the detection of regions of interest (ROI) in, e.g., attention mechanisms. In contrast to existing measures, the present approach handles situations of order uncertainties, where the order for some ROIs is crucial, while for others it is not. We compare the results of several measures in some theoretical cases as well as some real applications. We further demonstrate how our measure can be used to evaluate algorithms for ROI detection, particularly the model of Itti and Koch for bottom-up data-driven attention.

Are iterations and curvature useful for tensor voting

Tensor voting is an efficient algorithm for perceptual grouping and feature extraction, particularly for contour extraction. In this paper two studies on tensor voting are presented. First the use of iterations is investigated, and second, a new method for integrating curvature information is evaluated. In opposition to other grouping methods, tensor voting claims the advantage to be non-iterative. Although non-iterative tensor voting methods provide good results in many cases, the algorithm can be iterated to deal with more complex data configurations. The experiments conducted demonstrate that iterations substantially improve the process of feature extraction and help to overcome limitations of the original algorithm. As a further contribution we propose a curvature improvement for tensor voting. On the contrary to the curvature-augmented tensor voting proposed by Tang and Medioni, our method takes advantage of the curvature calculation already performed by the classical tensor voting and evaluates the full curvature, sign and amplitude. Some new curvature-modified voting fields are also proposed. Results show a lower degree of artifacts, smoother curves, a high tolerance to scale parameter changes and also more noise-robustness.

Iterated tensor voting and curvature improvement

Tensor voting (TV) methods have been developed in a series of papers by Medioni and coworkers during the last years. The method has been proved efficient for feature extraction and grouping and has been applied successfully in a diversity of applications such as contour and surface inferences, motion analysis, etc. We present here two studies on improvements of the method. The first one consists in iterating the TV process, and the second one integrates curvature information. In contrast to other grouping methods, TV claims the advantage to be non-iterative. Although non-iterative TV methods provide good results in many cases, the algorithm can be iterated to deal with more complex or more ambiguous data configurations. We present experiments that demonstrate that iterations substantially improve the process of feature extraction and help to overcome limitations of the original algorithm. As a further contribution, we propose a curvature improvement for TV. Unlike the curvature-augmented TV proposed by Tang and Medioni, our method evaluates the full curvature, sign and amplitude in the 2D case. Another advantage of the method is that it uses part of the curvature calculation already performed by the classical TV, limiting the computational costs. Curvature-modified voting fields are also proposed. Results show smoother curves, a lower degree of artifacts and a high tolerance against scale variations of the input. The methods are finally tested under noisy conditions showing that the proposed improvements preserve the noise robustness of the TV method.

A neural model of feature attention in motion perception

We utilize a model of motion perception to link a physiological study of feature attention in cortical motion processing to a psychophysical experiment of motion perception. We explain effects of feature attention by modulatory excitation of neural activity patterns in a framework of biased competition. Our model allows us to qualitatively replicate physiological data concerning attentional modulation and to generate model behavior in a decision experiment that is consistent with psychophysical observations. Furthermore, our investigation makes predictions for future psychophysical experiments.

Neural Architecture for Temporal Emotion Classification

In this pilot study, a neural architecture for temporal emotion recognition from image sequences is proposed. The investigation aims at the development of key principles in an extendable experimental framework to study human emotions. Features representing temporal facial variations were extracted within a bounding box around the face that is segregated into regions. Within each region, the optical flow is tracked over time. The dense flow field in a region is subsequently integrated whose principal components were estimated as a representative velocity of face motion. For each emotion a Fuzzy ARTMAP neural network was trained by incremental learning to classify the feature vectors resulting from the motion processing stage. Single category nodes corresponding to the expected feature representation code the respective emotion classes. The architecture was tested on the Cohn-Kanade facial expression database.