Volker Stephan

Generative character of perception: a neural architecture for sensorimotor anticipation

The basic idea of our anticipatory approach to perception is to avoid the common separation of perception and generation of behavior and to fuse both aspects into a consistent neural process. Our approach tries to explain the phenomenon of perception, in particular, of perception at the level of sensorimotor intelligence, from a behavior-oriented point of view. Perception is assumed to be a generative process of anticipating the course of events resulting from alternative sequences of hypothetically executed actions. By means of this sensorimotor anticipation, it is possible to characterize a visual scenery immediately in categories of behavior, i.e. by a set of actions which describe possible methods of interaction with the objects in the environment. Thus, the competence to perceive a complex situation can be understood as the capability to anticipate the course of events caused by different action sequences. Starting from an abstract description of anticipatory perception and the essential biological evidence for internal simulation, we present two biologically motivated computational models that are able to anticipate and evaluate hypothetically sensorimotor sequences. Both models consider functional aspects of those cortical and subcortical systems that are assumed to be involved in the process of sensory prediction and sensorimotor control. Our first approach, the Model for Anticipation based on Sensory IMagination (MASIM), realizes a sequential search in sensorimotor space using a simple model of lateral cerebellum as sensory predictor. We demonstrate the efficiency of this model approach in the light of visually guided local navigation behaviors of a mobile system. The second approach, the Model for Anticipation based on Cortical Representations (MACOR), is actually still at a conceptual level of realization. We postulate that this model allows a completely parallel search at the neocortical level using assemblies of spiking neurons for grouping, separation, and selection of sensorimotor sequences. Both models are intended as general schemes for anticipation based perception at the level of sensorimotor intelligence.

A neural field approach to topological reinforcement learning in continuous action spaces

We present a neural field approach to distributed Q-learning in continuous state and action spaces that is based on action coding and selection in dynamic neural fields. It is, to the best of our knowledge, one of the first attempts that combines the advantages of a topological action coding with a distributed action-value learning in one neural architecture. This combination, supplemented by a neural vector quantization technique for state space clustering, is the basis for a control architecture and learning scheme that meet the demands of reinforcement learning for real-world problems. The experimental results in learning a vision-based docking behavior, a hard delayed reinforcement learning problem, show that the learning process can be successfully accelerated and made robust by this kind of distributed reinforcement learning.

A NEW CONTROL SCHEME FOR COMBUSTION PROCESSES USING REINFORCEMENT LEARNING BASED ON NEURAL NETWORKS

We present a new control scheme for an industrial hard-coal combustion process in a power plant based on reinforcement-learning in combination with neural networks. To comply with the great requirements for environmental protection, the plant operator is interested in a minimization of the nitrogen oxides emission and a maximization of the efficiency factor, while other process parameters have to be kept within predefined limits. To cope with both the tremendous action and state space of the power plant, we present a multiagent-reinforcement-system consisting of 4 agents, which are realized by relatively simple neural function approximators. We demonstrate that our multiagent-system was able to significantly reduce the overall air consumption of the real combustion process of the power plant.

An efficient search strategy for feature selection using Chow-Liu trees

Within the taxonomy of feature extraction methods, recently the Wrapper approaches lost some popularity due to the associated computational burden, compared to Embedded or Filter methods. The dominating factor in terms of computational costs is the number of adaption cycles used to train the black box classifier or function approximator, e.g. a Multi Layer Perceptron. To keep a wrapper approach feasible, the number of adaption cycles has to be minimized, without increasing the risk of missing important feature subset combinations. We propose a search strategy, that exploits the interesting properties of Chow-Liu trees to reduce the number of considered subsets significantly. Our approach restricts the candidate set of possible new features in a forward selection step to children from certain tree nodes. We compare our algorithm with some basic and well known approaches for feature subset selection. The results obtained demonstrate the efficiency and effectiveness of our method.

Neural anticipative architecture for expectation driven perception

In this paper we present a biologically inspired neural architecture for visual perception based on anticipation. The main goal of this work is to demonstrate, that anticipation is a central key to improve the perception performance of technical systems. The presented approach is able to increase the robustness of the perception process against noise or sensory dropouts. We demonstrate these perceptional improvements through our architecture at the level of local navigation behavior of the miniature robot Khepera. We claim that perception is not an end in itself. Instead it is a sensorimotor process integrating the generation of behavior.

A reinforcement learning based neural multiagent system for control of a combustion process

We present a control scheme based on reinforcement learning for an industrial hard-coal combustion process in a power plant. To comply with the great demands on environmental protection, the plant operator is interested in a minimization of the nitrogen oxides emission, while other process parameters have to be kept within predefined limits. To cope with both the tremendous action and situation space of the power plant, we present a multiagent reinforcement system consisting of 4 agents, which are realized by relatively simple neural function approximators. We demonstrate, that our multiagent system was able to significantly reduce the overall air consumption of the real combustion process of the power plant.

Extension of the ALVINN-architecture for robust visual guidance of a miniature robot

Extensions of the ALVINN architecture are introduced for a KHEPERA miniature robot to navigate visually robust in a labyrinth. The reimplementation of the ALVINN-approach demonstrates, that also in indoor-environments a complex visual robot navigation is achievable using a direct input-output-mapping with a multilayer perceptron network, which is trained by expert-cloning. With the extensions it succeeds to overcome the restrictions of the small visual field of the camera by completing the input vector with history-components, introduction of the velocity dimension and evaluation of the networks output by a dynamic neural field. This creates the prerequisites to take turns which are no longer visible in the actual image and so make use of several alternatives of actions.

Machine learning techniques for selforganizing combustion control

This paper presents the overall system of a learning, selforganizing, and adaptive controller used to optimize the combustion process in a hard-coal fired power plant. The system itself identifies relevant channels from the available measurements, classical process data and flame image information, and selects the most suited ones to learn a control strategy based on observed data. Due to the shifting nature of the process, the ability to re-adapt the whole system automatically is essential. The operation in a real power plant demonstrates the impact of this intelligent control system with its ability to increase efficiency and to reduce emissions of greenhouse gases much better then any previous control system.

Fast and robust prediction of optical flow field sequences for visuomotor anticipation

In this paper, we present a hybrid neural architecture to predict optical flow fields as consequences of real and hypothetical actions. In this architecture, we introduce a neural field-based method to fuse sensory bottom-up and predicted top-down expectations. All subsystems extensively use confidence estimations to reduce disturbances caused by noise. The facilities of this anticipative preprocessing can be demonstrated by means of an optical flow field based local navigation behavior of the miniature robot KHEPERA. Our anticipative preprocessing enables the robot to bridge gaps of sensory dropouts and, in consequence, to avoid collisions even with very noisy sensory information.

Adaptive Feature Transformation for Image Data from Non-stationary Processes

This paper introduces the application of the feature transformation approach proposed by Torkkola [1] to the domain of image processing. Thereto, we extended the approach and identifed its advantages and limitations. We compare the results with more common transformation methods like Principal Component Analysis and Linear Discriminant Analysis for a function approximation task from the challenging domain of video-based combustion optimization. It is demonstrated that the proposed method generates superior results in very low dimensional subspaces. Further, we investigate the usefulness of an adaptive variant of the introduced method in comparison to basic subspace transformations and discuss the results.