Heiko Neumann

Disambiguating Visual Motion Through Contextual Feedback Modulation

Motion of an extended boundary can be measured locally by neurons only orthogonal to its orientation (aperture problem) while this ambiguity is resolved for localized image features, such as corners or nonocclusion junctions. The integration of local motion signals sampled along the outline of a moving form reveals the object velocity. We propose a new model of V1-MT feedforward and feedback processing in which localized V1 motion signals are integrated along the feedforward path by model MT cells. Top-down feedback from MT cells in turn emphasizes model V1 motion activities of matching velocity by excitatory modulation and thus realizes an attentional gating mechanism. The model dynamics implement a guided filling-in process to disambiguate motion signals through biased on-center, off-surround competition. Our model makes predictions concerning the time course of cells in area MT and V1 and the disambiguation process of activity patterns in these areas and serves as a means to link physiological mechanisms with perceptual behavior. We further demonstrate that our model also successfully processes natural image sequences.

The Role of Attention in Figure-Ground Segregation in Areas V1 and V4 of the Visual Cortex

Our visual system segments images into objects and background. Figure-ground segregation relies on the detection of feature discontinuities that signal boundaries between the figures and the background and on a complementary region-filling process that groups together image regions with similar features. The neuronal mechanisms for these processes are not well understood and it is unknown how they depend on visual attention. We measured neuronal activity in V1 and V4 in a task where monkeys either made an eye movement to texture-defined figures or ignored them. V1 activity predicted the timing and the direction of the saccade if the figures were task relevant. We found that boundary detection is an early process that depends little on attention, whereas region filling occurs later and is facilitated by visual attention, which acts in an object-based manner. Our findings are explained by a model with local, bottom-up computations for boundary detection and feedback processing for region filling.

Recurrent V1–V2 interaction in early visual boundary processing

Abstract. A majority of cortical areas are connected via feedforward and feedback fiber projections. In feedforward pathways we mainly observe stages of feature detection and integration. The computational role of the descending pathways at different stages of processing remains mainly unknown. Based on empirical findings we suggest that the top-down feedback pathways subserve a context-dependent gain control mechanism. We propose a new computational model for recurrent contour processing in which normalized activities of orientation selective contrast cells are fed forward to the next processing stage. There, the arrangement of input activation is matched against local patterns of contour shape. The resulting activities are subsequently fed back to the previous stage to locally enhance those initial measurements that are consistent with the top-down generated responses. In all, we suggest a computational theory for recurrent processing in the visual cortex in which the significance of local measurements is evaluated on the basis of a broader visual context that is represented in terms of contour code patterns. The model serves as a framework to link physiological with perceptual data gathered in psychophysical experiments. It handles a variety of perceptual phenomena, such as the local grouping of fragmented shape outline, texture surround and density effects, and the interpolation of illusory contours.

Multiple classifier systems for the classificatio of audio-visual emotional states

Research activities in the field of human-computer interaction increasingly addressed the aspect of integrating some type of emotional intelligence. Human emotions are expressed through different modalities such as speech, facial expressions, hand or body gestures, and therefore the classification of human emotions should be considered as a multimodal pattern recognition problem. The aim of our paper is to investigate multiple classifier systems utilizing audio and visual features to classify human emotional states. For that a variety of features have been derived. From the audio signal the fundamental frequency, LPCand MFCC coefficients, and RASTA-PLP have been used. In addition to that two types of visual features have been computed, namely form and motion features of intermediate complexity. The numerical evaluation has been performed on the four emotional labels Arousal, Expectancy, Power, Valence as defined in the AVEC data set. As classifier architectures multiple classifier systems are applied, these have been proven to be accurate and robust against missing and noisy data.

Neural mechanisms of cortico-cortical interaction in texture boundary detection: a modeling approach.

Texture information is an elementary feature utilized by the human visual system to automatically, or pre-attentively, segment the visual scene. The neural substrate underlying human texture processing as well as the basic computational mechanisms remains largely unknown up to now. We propose a neural model of texture processing which integrates the data obtained by a variety of methods into a common computational framework. It consists of a hierarchy of bi-directionally linked visual areas each containing topographical maps of mutually interconnected cells. It builds upon the two key hypotheses that (i). texture segmentation is based on boundary detection and that (ii). texture border detection is mainly a function of higher visual cortical areas such as V4. This model, while attempting to explain the processing of textures, is embedded in a more general neural model architecture of the infero-temporal pathway of form processing.The model allows to link human performance in texture segmentation with model cell activation patterns, in turn permitting to trace back fundamental psychophysical results on texture processing to their putative neural origins. Most importantly, it enables us to identify and evaluate the functional role of feedback connections between cortical areas in the context of texture processing, namely the suppression of ambiguous cell activities leading to a sharply localized detection of texture boundaries. One of the likely neural origins of modulatory effects on V1 cell activation levels, as observed in electrophysiological studies using single- and multi-unit recordings, can be resolved.

Luminance and edge information in grouping: a study using visual search

Preattentive grouping is supported by 2 systems, a brightness system that is contrast polarity sensitive and an edge system that is relatively insensitive to contrast polarity. Search was spatially parallel for pairs of same contrast polarity vertically aligned circles, among horizontal pairs, and serial for pairs of circles that had the opposite contrast polarity (Experiments 1-3). By replacing the circles with squares, the authors investigated the effect of adding collinear edge information. When collinear edges were present, the polarity difference between paired items did not disrupt grouping (Experiments 4-6). These results support models of grouping in which brightness and edge information are processed separately (e.g., S. Grossberg & E. Mingolla, 1985) and models of visual search in which complex relations between stimuli can be computed in parallel across the display.

Perception and Interactive Technologies

Head Pose and Eye Gaze Tracking.- Guiding Eye Movements for Better Communication and Augmented Vision.- Detection of Head Pose and Gaze Direction for Human-Computer Interaction.- Modelling and Simulation of Perception.- Modelling and Simulation of Spontaneous Perception Switching with Ambiguous Visual Stimuli in Augmented Vision Systems.- Neural Network Architecture for Modeling the Joint Visual Perception of Orientation, Motion, and Depth.- Integrating Information from Multiple Channels.- AutoSelect: What You Want Is What You Get: Real-Time Processing of Visual Attention and Affect.- Emotion Recognition Using Physiological and Speech Signal in Short-Term Observation.- Visual and Auditory Displays Driven by Perceptive Principles.- Visual Attention in Auditory Display.- A Perceptually Optimized Scheme for Visualizing Gene Expression Ratios with Confidence Values.- Spoken Dialogue Systems.- Combining Speech User Interfaces of Different Applications.- Learning and Forgetting of Speech Commands in Automotive Environments.- Help Strategies for Speech Dialogue Systems in Automotive Environments.- Multimodal and Situated Dialogue Systems.- Information Fusion for Visual Reference Resolution in Dynamic Situated Dialogue.- Speech and 2D Deictic Gesture Reference to Virtual Scenes.- Combining Modality Theory and Context Models.- Integration of Perceptive Technologies and Animation.- Visual Interaction in Natural Human-Machine Dialogue.- Multimodal Sensing, Interpretation and Copying of Movements by a Virtual Agent.- Poster Session.- Perception of Dynamic Facial Expressions of Emotion.- Multi-level Face Tracking for Estimating Human Head Orientation in Video Sequences.- The Effect of Prosodic Features on the Interpretation of Synthesised Backchannels.- Unsupervised Learning of Spatio-temporal Primitives of Emotional Gait.- System Demonstrations.- Talking with Higgins: Research Challenges in a Spoken Dialogue System.- Location-Based Interaction with Children for Edutainment.- An Immersive Game - Augsburg Cityrun.- Gaze-Contingent Spatio-temporal Filtering in a Head-Mounted Display.- A Single-Camera Remote Eye Tracker.- Miniature 3D TOF Camera for Real-Time Imaging.

Quantification of human body fat tissue percentage by MRI

The MRI‐based evaluation of the quantity and regional distribution of adipose tissue is one objective measure in the investigation of obesity. The aim of this article was to report a comprehensive and automatic analytical method for the determination of the volumes of subcutaneous fat tissue (SFT) and visceral fat tissue (VFT) in either the whole human body or selected slices or regions of interest. Using an MRI protocol in an examination position that was convenient for volunteers and patients with severe diseases, 22 healthy subjects were examined. The software platform was able to merge MRI scans of several body regions acquired in separate acquisitions. Through a cascade of image processing steps, SFT and VFT volumes were calculated. Whole‐body SFT and VFT distributions, as well as fat distributions of defined body slices, were analysed in detail. Complete three‐dimensional datasets were analysed in a reproducible manner with as few operator‐dependent interventions as possible. In order to determine the SFT volume, the ARTIS (Adapted Rendering for Tissue Intensity Segmentation) algorithm was introduced. The advantage of the ARTIS algorithm was the delineation of SFT volumes in regions in which standard region grow techniques fail. Using the ARTIS algorithm, an automatic SFT volume detection was feasible. MRI data analysis was able to determine SFT and VFT volume percentages using new analytical strategies. With the techniques described, it was possible to detect changes in SFT and VFT percentages of the whole body and selected regions. The techniques presented in this study are likely to be of use in obesity‐related investigations, as well as in the examination of longitudinal changes in weight during various medical conditions. Copyright

A Fast Biologically Inspired Algorithm for Recurrent Motion Estimation

We have previously developed a neurodynamical model of motion segregation in cortical visual area V1 and MT of the dorsal stream. The model explains how motion ambiguities caused by the motion aperture problem can be solved for coherently moving objects of arbitrary size by means of cortical mechanisms. The major bottleneck in the development of a reliable biologically inspired technical system with real-time motion analysis capabilities based on this neural model is the amount of memory necessary for the representation of neural activation in velocity space. We propose a sparse coding framework for neural motion activity patterns and suggest a means by which initial activities are detected efficiently. We realize neural mechanisms such as shunting inhibition and feedback modulation in the sparse framework to implement an efficient algorithmic version of our neural model of cortical motion segregation. We demonstrate that the algorithm behaves similarly to the original neural model and is able to extract image motion from real world image sequences. Our investigation transfers a neuroscience model of cortical motion computation to achieve technologically demanding constraints such as real-time performance and hardware implementation. In addition, the proposed biologically inspired algorithm provides a tool for modeling investigations to achieve acceptable simulation time

A recurrent model of contour integration in primary visual cortex

Physiological and psychophysical studies have demonstrated the importance of colinearity in visual processing. Motivated by these empirical findings we present a novel computational model of recurrent long-range processing in the primary visual cortex. Unlike other models we restrict the long-range interaction to cells of parallel orientation with colinear aligned receptive fields. We also employ a recurrent interaction using modulatory feedback, in accordance with empirical findings. Self-normalizing shunting equations guarantee the saturation of activities after a few recurrent cycles. The primary computational goal of the model is to evaluate local, often noisy orientation measurements within a more global context and to selectively enhance coherent activity by excitatory, modulating feedback. All model simulations were done with the same set of parameters. We show that the model qualitatively reproduces empirical data of response facilitation and suppression for a single bar element depending on the local surround outside the classical receptive field (M. K. Kapadia, M. Ito, C. D. Gilbert, & G. Westheimer, 1995). Next we evaluate the model performance for the processing of artificial and natural images. We quantitatively evaluate the model using two measures of contour saliency and orientation significance. We show that both measures monotonically increase during the recurrent interaction and saturate after a small number of recurrent cycles. The model clarifies how basic tasks of early vision can be accomplished within a single, biologically plausible architecture.