Jörg Bruske

Dynamic cell structure learns perfectly topology preserving map

Dynamic cell structures (DCS) represent a family of artificial neural architectures suited both for unsupervised and supervised learning. They belong to the recently (Martinetz 1994) introduced class of topology representing networks (TRN) that build perfectly topology preserving feature maps. DCS employ a modified Kohonen learning rule in conjunction with competitive Hebbian learning. The Kohonen type learning rule serves to adjust the synaptic weight vectors while Hebbian learning establishes a dynamic lateral connection structure between the units reflecting the topology of the feature manifold. In case of supervised learning, i.e., function approximation, each neural unit implements a radial basis function, and an additional layer of linear output units adjusts according to a delta-rule. DCS is the first RBF-based approximation scheme attempting to concurrently learn and utilize a perfectly topology preserving map for improved performance. Simulations on a selection of CMU-Benchmarks indicate that the DCS idea applied to the growing cell structure algorithm (Fritzke 1993c) leads to an efficient and elegant algorithm that can beat conventional models on similar tasks.

Intrinsic dimensionality estimation with optimally topology preserving maps

A new method for analyzing the intrinsic dimensionality (ID) of low-dimensional manifolds in high-dimensional feature spaces is presented. Compared to a previous approach by Fukunaga and Olsen (1971), the method has only linear instead of cubic time complexity with respect to the dimensionality of the input space. Moreover, it is less sensitive to noise than the former approach. Experiments include ID estimation of synthetic data for comparison and illustration as well as ID estimation of an image sequence.

Biologically inspired calibration-free adaptive saccade control of a binocular camera-head

Abstract. This paper describes fast and accurate calibration-free adaptive saccade control of a four-degrees-of-freedom binocular camera-head by means of Dynamic Cell Structures (DCS). The approach has been inspired by biology because primates face a similar problem and there is strong evidence that they have solved it in a similar way, i.e., by error feedback learning of an inverse model. Yet the emphasis of this article is not on detailed biological modeling but on how incremental growth of our artificial neural network model up to a prespecified precision results in very small networks suitable for real-time saccade control. Error-feedback-based training of this network proceeds in two phases. In the first phase we use a crude model of the cameras and the kinematics of the head to learn the topology of the input manifold together with a rough approximation of the control function off-line. In contrast to, for example, Kohonen-type adaptation rules, the distribution of neural units minimizes the control error and does not merely mimic the input probability density. In the second phase, the operating phase, the linear output units of the network continue to adapt on-line. Besides our TRC binocular camera-head we use a Datacube image processing system and a Stäubli R90 robot arm for automated training in the second phase. It will be demonstrated that the controller successfully corrects errors in the model and rapidly adapts to changing parameters.

An integrated architecture for learning of reactive behaviors based on dynamic cell structures

Abstract In this contribution we want to draw the readers attention to the advantages of dynamic cell structures (DCSs) (Bruske and Sommer, 1995) for learning reactive behaviors of autonomous robots. These include incremental on-like learning, fast output calculation , a flexible integration of different learning rules and a close connection to fuzzy logic . The latter allows for incorporation of prior knowledge and to interpret learning with DCSs as fuzzy rule generation and adaptation . After successful applications of DCSs to tasks involving supervised learning, feedback error learning and incremental category learning, in this article we take reinforcement learning of reactive collision avoidance for an autonomous mobile robot as a further example to demonstrate the validity of our approach. More specifically, we employ a REINFORCE (Williams, 1992) algorithm in combination with an adaptive heuristic critique (AHC) (Sutton, 1988) to learn a continuous valued sensory motor mapping for obstacle avoidance with a TRC Labmate from delayed reinforcement. The sensory input consists of eight unprocessed sonar readings, the controller output is the continuous angular and forward velocity of the Labmate. The controller and the AHC are integrated within a single DCS network, and the resulting avoidance behavior of the robot can be analyzed as a set of fuzzy rules, each rule having an additional certainty value .

Dynamic Cell Structures for the Evaluation of Keypoints in Facial Images

In this contribution Dynamic Cell Structures (DCS network) are applied to classify local image structures at particular facial landmarks. The facial landmarks such as the corners of the eyes or intersections of the iris with the eyelid are computed in advance by a combined model and data driven sequential search strategy. To reduce the detection error after the processing of the sequential search strategy, the computed image positions are verified applying a DCS network. The DCS network is trained by supervised learning with feature vectors which encode spatially arranged edge and structural information at the keypoint position considered. The model driven localization as well as the data driven verification are based on steerable filters, which build a representation comparable with one provided by a receptive field in the human visual system. We apply a DCS based classifier because of its ability to grasp the topological structure of complex input spaces and because it has proved successful in a number of other classification tasks. In our experiments the average error resulting from false positive classifications is less than 1%.

Topology Representing Networks for Intrinsic Dimensionality Estimation

In this paper we compare two methods for intrinsic dimensionality (ID) estimation based on optimally topology preserving maps (OTPMs). The first one is a direct approach, where the intrinsic dimensionality is estimated directly from the OTPM. We argue that this approach suffers from both practical and theoretical pitfalls. The second is a new approach which combines OTPMs with an efficient local principal component analysis (PCA). Exploiting the OTPM, local PCA can be shown to have only linear time complexity w.r.t. the dimensionality of the input space (in contrast to the prohibitive cubic complexity of the conventional approach), and hence the method becomes applicable even for very high dimensional input spaces as frequently encountered in computer vision. A local ID estimate is then obtained as the local number of significant eigenvalues. In addition to ID estimation the local subspaces as revealed by our local PCA can be directly used for further data processing tasks including classification and regression. The workability of the new approach for ID estimation and subspace auto-association is demonstrated on a sequence of 64 x 64 pixel images (4096-dimensional input space).

An Algorithm for Intrinsic Dimensionality Estimation

In this paper a new method for analyzing the intrinsic dimensionality (ID) of low dimensional manifolds in high dimensional feature spaces is presented. The basic idea is to first extract a low-dimensional representation that captures the intrinsic topological structure of the input data and then to analyze this representation, i.e. to estimate the intrinsic dimensionality. Compared to previous approaches based on 1ocal PCA the method has a number of important advantages: First, it can be shown to have only linear time complexity w.r.t. the dimensionality of the input space (in contrast to the cubic complexity of the conventional approach) and hence becomes applicable even for very high dimensional input spaces. Second, it is less sensitive to noise than former approaches, and, finally, the extracted representation can be directly used for further data processing tasks including auto-association and classification.

Adaptive Saccade Control of a Binocular Head with Dynamic Cell Structures

In this article we report how Dynamic Cell Structures (DCS) [1] can be utilized to learn fast and accurate saccade control of a four-degrees-of-freedom Binocular Head. We solve the order selection problem by incremental growing of a DCS network until the controller meets a pre-specified precision. Calculation of the controller output is very fast and suitable for realtime control since the resulting network is as small as possible and only the best matching unit and its topological neighbors are activated on presentation of an input stimulus. Training of the DCS is based on error feedback learning and proceeds in two phases. In the first phase we use a crude model of the cameras and the kinematics of the head to learn the topology of the input submanifold and a rough approximation off-line. In a second phase, the operating phase, we employ error feedback learning for online adaptation of the linear output units. Besides our TRC binocular head we use a Datacube image processing system and a Staubli R90 robot arm for automated training in the second phase. The controller is demonstrated to successfully correct errors in the model and to rapidly adapt to changing parameters.

Dynamische Zellstrukturen Theorie und Anwendung eines KNN-Modells

In dieser Arbeit [Bru98] wird ein neuartiges Kunstliches Neuronales Netz (KNN), die Dynamische Zellstruktur (DCS), vorgestellt, die das Approximationsverhalten von Netzwerken radialer Basisfunktionen (RBF-Netzen) mit der Reprasentationskraft topologieerhaltender Karten vereint. Die topologieerhaltenden Karten werden dabei zu einer im Vergleich zu herkommlichen RBF-Netzwerken effizienteren Ausgabeberechnung und Adaption genutzt. Daneben tragen sie zu einer Regularisierung des Lernproblems bei und bewirken durch effektive Projektion auf die Eingabemannigfaltigkeit eine verminderte Rauschsensitivitat. Durch inkrementelle Adaption sowohl der Parameter als auch der Modellordnung und der topologieerhaltenden Reprasentation der Eingabemannigfaltigkeit weist die DCS ein Hochstmas an Flexibilitat auf. Hierfur werden eine Vielzahl von Lernregeln angegeben, die im Kontext der stochastischen Approximation und der statistischen Mustererkennung diskutiert werden. Die DCS ist eine Weiterentwicklung von Fritzkes Wachsender Zellstruktur, [Fri95], an die sich ihre Namensgebung anlehnt.

Neural Fuzzy Techniques in Sonar-Based Collision Avoidance

In this chapter we report application of neuro-fuzzy control to sonar based collision avoidance of our TRC labmate robot, Figure 5. To this end, we will first provide the reader with a brief overview of existing concepts of neuro-fuzzy control and then present our own approach based on Radial Basis Functions. This particular Fuzzy-RBF (FRBF) approach is innovative w.r.t. three aspects of neuro-fuzzy control. First, it alleviates the covering problem in fuzzy control, i.e. the problem of an exponential growth of the number of rules with the dimension of the input space. Second, it provides a means for exact interpolation, i.e. inspite of overlapping membership functions the output of the controller can be guaranteed to take the value of the i-th rule if it has degree of fulfillment one. Finally, by using DCS, [1], instead of RBF networks, output calculation of the controller is very fast on average, since only a few rules (the best matching ones) are evaluated on presentation of an input to the controller.