Christian Faubel

2008 Special Issue: Learning to recognize objects on the fly: A neurally based dynamic field approach

Autonomous robots interacting with human users need to build and continuously update scene representations. This entails the problem of rapidly learning to recognize new objects under user guidance. Based on analogies with human visual working memory, we propose a dynamical field architecture, in which localized peaks of activation represent objects over a small number of simple feature dimensions. Learning consists of laying down memory traces of such peaks. We implement the dynamical field model on a service robot and demonstrate how it learns 30 objects from a very small number of views (about 5 per object are sufficient). We also illustrate how properties of feature binding emerge from this framework.

Dynamic Neural Fields as Building Blocks of a Cortex-Inspired Architecture for Robotic Scene Representation

Based on the concepts of dynamic field theory (DFT), we present an architecture that autonomously generates scene representations by controlling gaze and attention, creating visual objects in the foreground, tracking objects, reading them into working memory, and taking into account their visibility. At the core of this architecture are three-dimensional dynamic neural fields (DNFs) that link feature to spatial information. These three-dimensional fields couple into lower dimensional fields, which provide the links to the sensory surface and to the motor systems. We discuss how DNFs can be used as building blocks for cognitive architectures, characterize the critical bifurcations in DNFs, as well as the possible coupling structures among DNFs. In a series of robotic experiments, we demonstrate how the DNF architecture provides the core functionalities of a scene representation.

Anthropomorphism as a pervasive design concept for a robotic assistant

CORA is a robotic assistant whose task is to collaborate with a human operator on simple manipulation or handling tasks. Its sensory channels comprising vision, audition, haptics, and force sensing are used to extract perceptual information about speech, gestures and gaze of the operator, and object recognition. The anthropomorphic robot arm makes goal-directed movements to pick up and hand over objects. The human operator may mechanically interact with the arm by pushing it away (haptics) or by taking an object out of the robots gripper (force sensing). The design objective has been to exploit the human operators intuition by modeling the mechanical structure, the senses, and the behaviors of the assistant on human anatomy, human perception, and human motor behavior.

CORA: An anthropomorphic robot assistant for human environment

We describe the general concept, system architecture, hardware, and the behavioral abilities of CORA (Cooperative Robot Assistant), an autonomous nonmobile robot assistant. Outgoing from our basic assumption that the behavior to perform determines the internal and external structure of the behaving system, we have designed CORA anthropomorphic to allow for humanlike behavioral strategies in solving complex tasks. Although CORA was built as a prototype of a service robot system to assist a human partner in industrial assembly tasks, we will show that CORAs behavioral abilities are also conferrable in a household environment. After the description of the hardware platform and the basic concepts of our approach, we present some experimental results by means of an assembly task.

A neuro-dynamic architecture for one shot learning of objects that uses both bottom-up recognition and top-down prediction

Learning to recognize objects from a small number of example views is a difficult problem of robot vision, of particular importance to assistance robots who are taught by human users. Here we present an approach that combines bottom-up recognition of matching patterns and top-down estimation of pose parameters in a recurrent loop that improves on previous efforts to reconcile invariance of recognition under view changes with discrimination among different objects. We demonstrate and evaluate the approach both in a service robotics implementation as well as on the COIL database. The robotic implementation highlights features of our approach that enable real-time pose tracking as well as recognition from views where figure ground segmentation is difficult.

Making a robotic scene representation accessible to feature and label queries

We present a neural architecture for scene representation that stores semantic information about objects in the robots workspace. We show how this representation can be queried both through low-level features such as color and size, through feature conjunctions, as well as through symbolic labels. This is possible by binding different feature dimensions through space and integrating these space-feature representations with an object recognition system. Queries lead to the activation of a neural representation of previously seen objects, which can then be used to drive object-oriented action. The representation is continuously linked to sensory information and autonomously updates when objects are moved or removed.

Scenes and tracking with dynamic neural fields: How to update a robotic scene representation

We present an architecture based on the Dynamic Field Theory for the problem of scene representation. At the core of this architecture are three-dimensional neural fields linking feature to spatial information. These three-dimensional fields are coupled to lower-dimensional fields that provide both a close link to the sensory surface and a close link to motor behavior. We highlight the updating mechanism of this architecture, both when a single object is selected and followed by the robots head in smooth pursuit and in multi-item tracking when several items move simultaneously.

A neuro-dynamic object recognition architecture enhanced by foveal vision and a gaze control mechanism

We present an extension of a neuro-dynamic object recognition system that combines bottom-up recognition of matching patterns and top-down estimation of pose parameters in a recurrent loop. It is extended by an active foveal vision system. Adding the active vision component is easily integrated within the architecture and improves the recognition rate on previous experiments on the COIL-100 database and for scenes where segmentation of objects is not trivial. Furthermore the active component allows to substantially increase the spatial area where objects can be tracked. When objects move faster than visual servoing can track, catch-up saccades are autonomously generated.

Learning objects on the fly - object recognition for the here and now

We present a robotic vision system for object recognition, pose estimation and fast object learning. Our approach uses the Dynamic Neural Field Theory to combine bottom-up recognition of matching patterns and top-down estimation of pose parameters in a recurrent loop. Because Dynamic Neural Fields provide the system with stabilized percepts that still track changes in the incoming sensory stream, the system is able to do pose tracking even if objects are shortly occluded or distractor objects are moved into the scene.

Instance-Based Object Recognition with Simultaneous Pose Estimation Using Keypoint Maps and Neural Dynamics

We present a method for biologically-inspired object recognition with one-shot learning of object appearance. We use a computationally efficient model of V1 keypoints to select object parts with the highest information content and model their surroundings using simple colour features. This map-like representation is fed into a dynamical neural network which performs pose, scale and translation estimation of the object given a set of previously observed object views. We demonstrate the feasibility of our algorithm for cognitive robotic scenarios and evaluate classification performance on a dataset of household items.