Rüdiger Dillmann

Representation and constrained planning of manipulation strategies in the context of Programming by Demonstration

In Programming by Demonstration, a flexible representation of manipulation motions is necessary to learn and generalize from human demonstrations. In contrast to subsymbolic representations of trajectories, e.g. based on a Gaussian Mixture Model, a partially symbolic representation of manipulation strategies based on a temporal satisfaction problem with domain constraints is developed. By using constrained motion planning and a geometric constraint representation, generalization to different robot systems and new environments is achieved. In order to plan learned manipulation strategies the RRT-based algorithm by Stilman et al. is extended to consider, that multiple sets of constraints are possible during the extension of the search tree.

Feature Set Selection and Optimal Classifier for Human Activity Recognition

Human activity recognition is an essential ability for service robots and other robotic systems which are in interaction with human beings. To be proactive, the system must be able to evaluate the current state of the user it is dealing with. Also future surveillance systems will benefit from robust activity recognition if realtime constraints are met, allowing to automate tasks that have to be fulfilled by humans yet. In this paper, a thorough analysis of features and classifiers aimed at human activity recognition is presented. Based on a set of 10 activities, the use of different feature selection algorithms is evaluated, as well as the results different classifiers (SVMs, Neural Networks, Bayesian Classifiers) provide in this context. Also the interdependency between feature selection method and chosen classifier is investigated. Furthermore, the optimal number of features to be used for an activity is examined.

Advances in Robot Programming by Demonstration

Robot Programming by Demonstration (PbD) has been dealt with in the literature as a promising way to teach robots new skills in an intuitive way. In this paper we describe our current work in the field toward the implementation of PbD system which allows robots to learn continuously from human observation, build generalized representations of human demonstration and apply such representations to new situations.

Developing and analyzing intuitive modes for interactive object modeling

In this paper we present two approaches for intuitive interactive modelling of special object attributes by use of specific sensoric hardware. After a brief overview over the state of the art in interactive, intuitive object modeling, we motivate the modeling task by deriving the dierent object attributes that shall be modeled from an analysis of important interactions with objects. As an example domain, we chose the setting of a service robot in a kitchen. Tasks from this domain were used to derive important basic actions from which in turn the necessary object attributes were inferred. In the main section of the paper, two of the derived attributes are presented, each with an intuitive interactive modeling method. The object attributes to be modeled a restable object positions and movement restrictions for objects. Both of the intuitive interaction methods were evaluated with a group of test persons and the results are discussed. The paper ends with conclusions on the discussed results and a preview of future work in this area, in particular of potential applications.

Using spatial relations of objects in real world scenes for scene structuring and scene understanding

Given a room full of individual objects in a generic household scene, one can observe that the objects are mostly not placed randomly but in a certain order. Because of this each object can be described by the surrounding objects and the spatial relations to those objects. This paper presents several types of spatial relationships that can be deduced using object positions in 3D as well as an approach to retrieve these relations from real world scenes via annotation of colored 3D pointclouds gathered with a sensor. Finally, a way to use this data to make predictions about an unknown object based on its surrounding objects is presented.

Learning of generalized manipulation strategies in the context of Programming by Demonstration

In Programming by Demonstration, abstract manipulation knowledge has to be learned, that can be used by an autonomous robot system in different environments with arbitrary obstacles. In this work, manipulation strategies are learned by observation of a human teacher and represented as a flexible, constraint-based representation of the search space for motion planning. The learned manipulation strategy contains a large set of automatically generated features, which are generalized using additional demonstrations of the teacher. The generalized manipulation strategy is executed on a real bimanual anthropomorphic robot system in different environments with arbitrary obstacles using constrained motion planning.

Bridging the Gap of Abstraction for Probabilistic Decision Making on a Multi-Modal Service Robot.

This paper proposes a decision making and control supervision system for a multi-modal service robot. With partially observable Markov decision processes (POMDPs) utilized for scenario level decision making, the robot is able to deal with uncertainty in both observation and environment dynamics and can balance multiple, conflicting goals. By using a flexible task sequencing system for fine grained robot component coordination, complex sub-activities, beyond the scope of current POMDP solutions, can be performed. The sequencer bridges the gap of abstraction between abstract POMDP models and the physical world concerning actions, and in the other direction multi-modal perception is filtered while preserving measurement uncertainty and model-soundness. A realistic scenario for an autonomous, anthropomorphic service robot, including the modalities of mobility, multi-modal humanrobot interaction and object grasping, has been performed robustly by the system for several hours. The proposed filterPOMDP reasoner is compared with classic POMDP as well as MDP decision making and a baseline finite state machine controller on the physical service robot, and the experiments exhibit the characteristics of the different algorithms.

Reasoning for a multi-modal service robot considering uncertainty in human-robot interaction

This paper presents a reasoning system for a multi-modal service robot with human-robot interaction. The reasoning system uses partially observable Markov decision processes (POMDPs) for decision making and an intermediate level for bridging the gap of abstraction between multi-modal real world sensors and actuators on the one hand and POMDP reasoning on the other. A filter system handles the abstraction of multi-modal perception while preserving uncertainty and model-soundness. A command sequencer is utilized to control the execution of symbolic POMDP decisions on multiple actuator components. By using POMDP reasoning, the robot is able to deal with uncertainty in both observation and prediction of human behavior and can balance risk and opportunity. The system has been implemented on a multi-modal service robot and is able to let the robot act autonomously in modeled human-robot interaction scenarios. Experiments evaluate the characteristics of the proposed algorithms and architecture.

Learning of probabilistic grasping strategies using Programming by Demonstration

The planning of grasping motions is demanding due to the complexity of modern robot systems. In Programming by Demonstration, the observation of a human teacher allows to draw additional information about grasping strategies. Rosell showed, that the motion planning problem can be simplified by globally restricting the set of valid configurations to a learned subspace. In this work, the transformation of a humanoid grasping strategy to an anthropomorphic robot system is described by a probabilistic model, called variation model, in order to account for modeling and transformation errors. The variation model resembles a soft preference for grasping motions similar to the demonstration and therefore induces a non-uniform sampling distribution on the configuration space. The sampling distribution is used in a standard probabilistic motion planner to plan grasping motions efficiently for new objects in new environments.

Learning of Planning Models for Dexterous Manipulation Based on Human Demonstrations

In the human environment service robots have to be able to manipulate autonomously a large variety of objects in a workspace restricted by collisions with obstacles, self-collisions and task constraints. Planning enables the robot system to generalize predefined or learned manipulation knowledge to new environments. For dexterous manipulation tasks the manual definition of planning models is time-consuming and error-prone. In this work, planning models for dexterous tasks are learned based on multiple human demonstrations using a general feature space including automatically generated contact constraints, which are automatically relaxed to consider the correspondence problem. In order to execute the learned planning model with different objects, the contact location is transformed to given object geometry using morphing. The initial, overspecialized planning model is generalized using a previously described, parallelized optimization algorithm with the goal to find a maximal subset of task constraints, which admits a solution to a set of test problems. Experiments on two different, dexterous tasks show the applicability of the learning approach to dexterous manipulation tasks.