Hans-Joachim Böhme

An Approach to Multi-modal Human-Machine Interaction for Intelligent Service Robots

Abstract The paper describes a multi-modal scheme for human–robot interaction suited for a wide range of intelligent service robot applications. Operating in un-engineered, cluttered, and crowded environments, such robots have to be able to actively contact potential users in their surroundings and to offer their services in an appropriate manner. Starting from a real application scenario, the usage of a robot as mobile information kiosk in a home store, some reliable methods for vision-based interaction, sound analysis and speech output have been developed. These methods are integrated into a prototypical interaction cycle that can be assumed as a general approach to human–machine interaction. Experimental results demonstrate the strengths and weaknesses of the proposed methods.

A Multi-Modal System for Tracking and Analyzing Faces on a Mobile Robot

Abstract This paper describes a user detection system which employs a saliency system working on an omnidirectional camera delivering a rough and fast estimate of the position of a potential user. It consists of a vision (skin color) and a sonar based component, which are combined to make the estimate more reliable. To make the skin color detection robust under varying illumination conditions, it is supplied with an automatic white balance algorithm. The active vision head looks continuously in the direction of the salient region. Thus, a high resolution image can be grabbed and analyzed with a face detector.

Classification of face images for gender, age, facial expression, and identity

In this paper we compare two models for extracting features from face images and several neural classifiers for their applicability to classify gender, age, facial expression, and identity. These models are i) a description of face images by their projection on independent base images and ii) an Active Appearance Model which describes the shape and grey value variations of the face images. The extracted feature vectors are classified with Nearest Neighbor, MLP, RBF and LVQ networks, and classification results are compared.

Neural Architecture for Gesture-Based Human-Machine-Interaction

We present a neural architecture for gesture-based interaction between a mobile robot and human users. One crucial problem for natural interface techniques is the robustness under highly varying environmental conditions. Therefore, we propose a multiple cue approach for the localisation of a potential user in the operation field, followed by the aquisition and interpretaion of its gestural instructions. The whole approach is motivated in the context of a reliable operation scenario, but can be extended easily for other applications, such as videoconferencing.

Conception and realization of a multi-sensory interactive mobile office guide

This paper describes the conception and realization of a multi-sensory interactive mobile office guide. We designed a mobile robot based on a modified pioneer-2 robot, which is able to welcome visitors in our department and guide them to the desired staff member. The main components of the system are a vision based multi-person-tracker and the existing navigation toolkit CARMEN. Furthermore, the robot provides a bidirectional video conference system and a speech synthesis system. Experimental results show, that the implemented multi-person-tracker is accurately able to track an unknown number of persons in real-time and guide them to the respective people, while keeping an eye on the interaction partner.

Person Localization and Posture Recognition for Human-Robot Interaction

The development of a hybrid system for (mainly) gesture-based human-robot interaction is presented, thereby describing the progress in comparison to the work shown at the last gesture workshop (see [2]). The system makes use of standard image processing techniques as well as of neural information processing. The performance of our architecture includes the detection of a person as a potential user in an indoor environment, followed by the recognition of her gestural instructions. In this paper, we concentrate on two major mechanisms: (i), the contour-based person localization via a combination of steerable filters and three-dimensional dynamic neural fields, and (ii), our first experiences concerning the recognition of different instructional postures via a combination of statistical moments and neural classifiers.

Farb- und strukturbasierte neuronale Verfahren zur Lokalisierung von Gesichtern in Real-World-Szenen

Fur eine naturliche gestenbasierte Mensch-Maschine-Kommunikation in Real-World-Umgebungen ist die robuste Lokalisation eines potentiellen Nutzers eine elementare Voraussetzung. Dabei werden insbesondere Gesichter und Hande als besonders gestenrelevante Bildstrukturen angesehen. Der vorliegende Beitrag behandelt einen hautfarb- und einen strukturbasierten, neuronalen Lokalisationsmechanismus und stellt erste Ergebnisse zu beiden Ansatzen vor. Es wird verdeutlicht, das erst die Kombination der beiden, sich erganzenden Verfahren die erforderliche Robustheit der Nutzerlokalisation unter Real-World-Bedingungen sichert.

Extension of the ALVINN-architecture for robust visual guidance of a miniature robot

Extensions of the ALVINN architecture are introduced for a KHEPERA miniature robot to navigate visually robust in a labyrinth. The reimplementation of the ALVINN-approach demonstrates, that also in indoor-environments a complex visual robot navigation is achievable using a direct input-output-mapping with a multilayer perceptron network, which is trained by expert-cloning. With the extensions it succeeds to overcome the restrictions of the small visual field of the camera by completing the input vector with history-components, introduction of the velocity dimension and evaluation of the networks output by a dynamic neural field. This creates the prerequisites to take turns which are no longer visible in the actual image and so make use of several alternatives of actions.

A Neural Network Hierarchy for Data Driven and Knowledge Controlled Selective Visual Attention

We present a neural implementation of a dynamical network hierarchy for data driven and knowledge controlled selective visual attention. The model architecture is composed of several interacting subsystems for different processing tasks. With the example of real-world scene analysis the proposed model demonstrates its abilities in preattentive search and in decomposition of a complex visual input into a sequence of striking local input segments. Based on its functional architecture our model is able to shift its focus of attention both driven by the input data and controlled by its internal processing state and the already acquired knowledge.

Konturbasierte Personenlokalisation mittels dreidimensionaler neuronaler Felder und steuerbarer Filter

In dieser Arbeit wird ein Verfahren zur Lokalisation von Personen innerhalb unpraparierter visueller Szenen anhand typischer Konturen vorgestellt. Es wird sich dabei auf die ausere Kontur frontal ausgerichteter Personen im Bereich von Kopf und Schultern bezogen (Kopf-Schulter-Partie). Diese Kontur wird approximiert durch ein raumlich verteiltes Arrangement steuerbarer (steerable) Filter, das auf einer Anzahl von pyramidal abgestuften Auflosungen eines Bildes ange-wendet wird.