Alexander Koenig

TOOMAS: Interactive Shopping Guide robots in everyday use - final implementation and experiences from long-term field trials

The paper gives a comprehensive overview of our Shopping Guide project, which aims at the development of interactive mobile shopping companion robots for everyday use in challenging operating environments such as home improvement stores. It is spanning an arc from the expectations and requirements of store owners and customers, via the challenges of the shopping scenario and the operating environment, the implemented functionality of the shopping guide robots, up to the results of long-term field trials. The field trials started in April 2008 and still ongoing aim at studying whether and how a group of interactive mobile shopping guide robots can operate completely autonomously in such everyday environments and how they are accepted by uninstructed customers. In these field trials, where nine robotic shopping guides traveled together 2187 kilometers in three different home improvement stores in Germany, more than 8,600 customers were successfully guided to the locations of their products of choice. With the successful development of these shopping guide robots, a further important step towards assistive robotics for daily use has been done.

ShopBot: Progress in developing an interactive mobile shopping assistant for everyday use

The paper describes progress achieved in our long-term research project ShopBot, which aims at the development of an intelligent and interactive mobile shopping assistant for everyday use in shopping centers or home improvement stores. It is focusing on recent progress concerning two important methodological aspects: (i) the on-line building of maps of the operation area by means of advanced Rao-Blackwellized SLAM approaches using both sonar-based gridmaps as well as vision-based graph maps as representations, and (ii) a probabilistic approach to multi-modal user detection and tracking during the guidance tour. Experimental results of both the map building characteristics and the person tracking behavior achieved in an ordinary home improvement store demonstrate the reliability of both approaches. Moreover, we present first very encouraging results of long-term field trials which have been executed with three robotic shopping assistants in another home improvement store in Bavaria since March 2008. In this field test, the robots could demonstrate their suitability for this challenging real-world application, as well as the necessary user acceptance.

Vision-based Monte Carlo self-localization for a mobile service robot acting as shopping assistant in a home store

We present a novel omnivision-based robot localization approach which utilizes the Monte Carlo Localization (MCL), a Bayesian filtering technique based on a density representation by means of particles. The capability of this method to approximate arbitrary likelihood densities is a crucial property for dealing with highly ambiguous localization hypotheses as are typical for real-world environments. We show how omnidirectional imaging can be combined with the MCL-algorithm to globally localize and track a mobile robot given a taught graph-based representation of the operation area. In contrast to other approaches, the nodes of our graph are labeled with both visual feature vectors extracted from the omnidirectional image, and odometric data about the pose of the robot at the moment of the node insertion (position and heading direction). To demonstrate the reliability of our approach, we present first experimental results in the context of a challenging robotics application, the self-localization of a mobile service robot acting as shopping assistant in a very regularly structured, maze-like and crowded environment, a home store.

Omnivision-based probabilistic self-localization for a mobile shopping assistant continued

The basic idea of our omniview-based MCL approach and preliminary experimental results were presented in our previous paper [Proc. IROS 2002, pp. 256-262]. In continuing, this paper describes a number of methodical and technical improvements addressing challenges arising from the characteristics of our real-world application, the vision-based self-localization of a mobile robot that acts as a shopping assistant in the maze-like environment of a home store. To cope with highly variable illumination conditions, we present a reference-based correction approach that realizes a robust, automatic luminance stabilization and color adaptation already at the level of image formation. To deal with severe occlusions or disturbances of the omnidirectional image caused by, e.g. people standing near the robot or local illumination artifacts, we introduce a novel selective observation comparison method as prerequisite for a robust particle filter update. Further studies investigate the impact of the utilized observation model on the localization accuracy. The results of a series of localization experiments carried out in the home store confirm the robustness and superiority of our advanced, real-time approach.

A graph matching technique for an appearance-based, visual SLAM-approach using Rao-Blackwellized Particle Filters

In continuation of our previous work on visual, appearance-based localization in manually built maps in this paper we present a novel appearance-based, visual SLAM approach. The essential contribution of this work is, an adaptive sensor model which is estimated online and a graph matching scheme to evaluate the likelihood of a given topological map. Both methods enable the combination of an appearance-based, visual localization concept with a Rao-Blackwellized Particle Filter (RBPF) as state estimator to a real-world suitable, online SLAM approach. In our system, each RBPF particle incrementally constructs its own graph-based environment model which is labeled with visual appearance features (extracted from panoramic 360deg snapshots of the environment) and the estimated poses of the places where the snapshots were captured. The essential advantages of this appearance-based SLAM approach are its low memory and computing-time requirements. Therefore, the algorithm is able to perform in real-time. Finally, we present the results of SLAM experiments in two challenging environments that investigate the stability and localization accuracy of this SLAM technique.

Omniview-based concurrent map building and localization using adaptive appearance maps

This paper describes a novel omnivision-based concurrent map-building and localization (CML) approach which is able to robustly localize a mobile robot in a uniformly structured, maze-like environment with changing appearances. The presented approach extends and improves known appearance-based CML techniques in a few essential aspects. For example, an advanced learning scheme in combination with an active forgetting is introduced to allow a complexity restricting adaptation of the environment model to appearance variations of the operation area. Moreover, a generalized scheme for fusion of localization hypotheses from several state estimators with different meaning and certainty and a distributed coding of the current observation by a weighted set of reference observations is proposed. Finally, several real-world localization experiments investigating the stability and localization accuracy of this novel omnivision-based CML technique for a highly dynamic and populated operation area, a home store, are presented.

Robust omniview-based probabilistic self-localization for mobile robots in large maze-like environments

This paper extends our previous work on omniview-based Monte Carlo localization. It presents a number of improvements addressing challenges arising from the characteristics of the given real-world application, the self-localization of a mobile robot in a regularly structured, maze-like and populated operation area, a home store. The contribution of this paper can be summarized as follows: we introduce a more specific extraction of color-based appearance features and propose a novel selective observation comparison method to determine the similarity between expected and actual observation allowing a better handling of severe occlusions or disturbances. Moreover, we present the results of a series of localization experiments studying the impact of the appearance-feature extraction and the observation comparison on the localization accuracy. Our improved approach can successfully demonstrate its omniview-based localization capabilities for a demanding, large operation area - a home store with a size up to 100/spl times/60 m/sup 2/. To the best of our knowledge, this is the most complex operation area that has been studied experimentally so far using appearance-based localization techniques.

Neural architecture for concurrent map building and localization using adaptive appearance maps

This paper describes a novel omnivision-based Concurrent Map-building and Localization (CML) approach which is able to localize a mobile robot in complex and dynamic environments. The approach extends or improves known CML techniques in essential aspects. For example, a more flexible model of the environment is used to represent experienced observations. By applying an improved learning regime, observations which are not longer of importance for the localization task are actively forgotten to limit complexity. Furthermore, a generalized scheme for hypotheses fusion is presented that enables the integration of further multi-sensory position estimators.