Peter Kaiser

Extracting whole-body affordances from multimodal exploration

Humanoid robots that have to operate in cluttered and unstructured environments, such as man-made and natural disaster scenarios, require sophisticated sensorimotor capabilities. A crucial prerequisite for the successful execution of whole-body locomotion and manipulation tasks in such environments is the perception of the environment and the extraction of associated environmental affordances, i.e. the action possibilities of the robot in the environment, in order to generate whole-body locomotion and manipulation actions. We believe that such a coupling between perception and action could be a key to substantially increase the flexibility of humanoid robots. In this paper, we present an approach for the generation of whole-body locomotion and manipulation actions based on the affordances associated with environmental elements in the scene which are extracted via multimodal exploration. Based on the properties of detected environmental primitives and the estimated empty space in the scene, we propose methods to generate hypotheses for feasible whole-body actions while taking into account additional task constraints such as manipulability and balance. We combine visual and inertial sensing modalities by means of a novel depth model for generating segmented and categorized geometric primitives. A rule-based system is then incorporated to assign affordance hypotheses to these primitives. Finally, precomputed whole-body manipulability and stability maps are used for filtering affordances that are out of reach and for identifying the most promising locations for the action execution. We tested the developed methods in different scenes, unknown to the robot, demonstrating how reasonable the generated affordance hypotheses are.

Extraction of Whole-Body Affordances for Loco-Manipulation Tasks

Humanoid robots that have to operate in cluttered and unstructured environments, such as man-made and natural disaster scenarios, require sophisticated sensorimotor capabilities. A crucial prerequisite for the successful execution of whole-body locomotion and manipulation tasks in such environments is the perception of the environment and the extraction of associated environmental affordances, i.e., the action possibilities of the robot in the environment. We believe that such a coupling between perception and action could be a key to substantially increase the flexibility of humanoid robots. In this paper, we approach the affordance-based generation of whole-body actions for stable locomotion and manipulation. We incorporate a rule-based system to assign affordance hypotheses to visually perceived environmental primitives in the scene. These hypotheses are then filtered using extended reachability maps that carry stability information, for identifying reachable affordance hypotheses. We then formulate the hypotheses in terms of a constrained inverse kinematics problem in order to find whole-body configurations that utilize a chosen set of hypotheses. The proposed methods are implemented and tested in simulated environments based on RGB-D scans as well as on a real robotic platform.

Grounded spatial symbols for task planning based on experience

Providing autonomous humanoid robots with the abilities to react in an adaptive and intelligent manner involves low level control and sensing as well as high level reasoning. However, the integration of both levels still remains challenging due to the representational gap between the continuous state space on the sensorimotor level and the discrete symbolic entities used in high level reasoning. In this work, we approach the problem of learning a representation of the space which is applicable on both levels. This representation is grounded on the sensorimotor level by means of exploration and on the language level by making use of common sense knowledge. We demonstrate how spatial knowledge can be extracted from these two sources of experience. Combining the resulting knowledge in a systematic way yields a solution to the grounding problem which has the potential to substantially decrease the learning effort.

Validation of whole-body loco-manipulation affordances for pushability and liftability

Autonomous robots that are intended to work in disaster scenarios like collapsed or contaminated buildings need to be able to efficiently identify action possibilities in unknown environments. This includes the detection of environmental elements that allow interaction, such as doors or debris, as well as the utilization of fixed environmental structures for stable whole-body loco-manipulation. Affordances that refer to whole-body actions are especially valuable for humanoid robots as the necessity of stabilization is an integral part of their control strategies. Based on our previous work we propose to apply the concept of affordances to actions of stable whole-body loco-manipulation, in particular to pushing and lifting of large objects. We extend our perceptual pipeline in order to build large-scale representations of the robots environment in terms of environmental primitives like planes, cylinders and spheres. A rule-based system is employed to derive whole-body affordance hypotheses from these primitives, which are then subject to validation by the robot. An experimental evaluation demonstrates our progress in detection, validation and utilization of whole-body affordances.

Extracting common sense knowledge from text for robot planning

Autonomous robots often require domain knowledge to act intelligently in their environment. This is particularly true for robots that use automated planning techniques, which require symbolic representations of the operating environment and the robots capabilities. However, the task of specifying domain knowledge by hand is tedious and prone to error. As a result, we aim to automate the process of acquiring general common sense knowledge of objects, relations, and actions, by extracting such information from large amounts of natural language text, written by humans for human readers. We present two methods for knowledge acquisition, requiring only limited human input, which focus on the inference of spatial relations from text. Although our approach is applicable to a range of domains and information, we only consider one type of knowledge here, namely object locations in a kitchen environment. As a proof of concept, we test our approach using an automated planner and show how the addition of common sense knowledge can improve the quality of the generated plans.

RDT + : A parameter-free algorithm for exact motion planning

In this paper parameter-free concepts for exact motion planning are investigated. With the proposed RDT+ approach the collision detection parameters of a Rapidly-exploring Dense Tree (RDT) are automatically adjusted until an exact solution can be found. For efficient planning discrete collision detection routines are used within the RDT planner and by verifying the results with exact collision detection methods, the RDT+ concept allows to compute motions that are guaranteed collision-free. We show the probabilistic completeness of the proposed planner and present an extension for handling narrow passages. The algorithms are evaluated in different experiments, including narrow passages and high-dimensional planning problems, that are solved in simulation and on the humanoid robot ARMAR-III.

Towards a hierarchy of loco-manipulation affordances

We propose a formalism for the hierarchical representation of affordances. Starting with a perceived model of the environment consisting of geometric primitives like planes or cylinders, we define a hierarchical system for affordance extraction whose foundation are elementary power grasp affordances. Higher-level affordances, e.g. bimanual affordances, result from combining lower-level affordances with additional properties concerning the underlying geometric primitives of the scene. We model affordances as continuous certainty functions taking into account properties of the environmental elements and the perceiving robots embodiment. The developed formalism is regarded as the basis for the description of whole-body affordances, i.e. affordances associated with whole-body actions. The proposed formalism was implemented and experimentally evaluated in multiple scenarios based on RGB-D camera data. The feasibility of the approach is demonstrated on a real robotic platform.

IK-Map: An enhanced workspace representation to support inverse kinematics solvers

We present an approach to improve the performance of general purpose inverse kinematics (IK) solvers which are based on iterative gradient descent algorithms. The proposed IK-Map is used to represent the whole workspace of the manipulator through a voxelized data structure, similar to existing approaches, e.g. reachability or capability maps. We extend the reachability map approach by additionally storing reference IK solutions, which can be used to seed iterative IK solvers during online processing. This information can be used to limit the effect of well-known issues with local optimization schemes based on gradient decent methods, such as local minima or constraint violation. We evaluate the approach with a simulated model of ARMAR-4, showing that classical generic Jacobian-based IK solvers can be improved in terms of success rate, performance, and quality of the resulting IK solutions.

Constellation - An algorithm for finding robot configurations that satisfy multiple constraints

Planning motion for humanoid robots requires obeying simultaneous constraints on balance, collision-avoidance, and end-effector pose, among others. Several algorithms are able to generate configurations that satisfy these constraints given a good initial guess, i.e. a configuration which is already close to satisfying the constraints. However, when selecting goals for a planner a close initial guess is rarely available. Methods that attempt to satisfy all constraints through direct projection from a distant initial guess often fail due to opposing gradients for the various constraints, joint-limits, or singularities. We approach the problem of generating a constrained goal by searching for a configuration in the intersection of all constraint manifolds in configuration space (C-space). Starting with an initial guess, our algorithm, Constellation, builds a graph in C-space whose nodes are configurations that satisfy one or more constraints and whose cycles determine where the algorithm explores next. We compare the performance of our approach to direct projection and a previously-proposed cyclic projection method on reaching tasks for a humanoid robot with 33 DOF. We find that Constellation performs the best in terms of the number of solved queries across a wide range of problem difficulty. However, this success comes at higher computational cost.

Integrating multi-purpose natural language understanding, robot’s memory, and symbolic planning for task execution in humanoid robots

Abstract We propose an approach for instructing a robot using natural language to solve complex tasks in a dynamic environment. In this study, we elaborate on a framework that allows a humanoid robot to understand natural language, derive symbolic representations of its sensorimotor experience, generate complex plans according to the current world state, and monitor plan execution. The presented development supports replacing missing objects and suggesting possible object locations. It is a realization of the concept of structural bootstrapping developed in the context of the European project Xperience. The framework is implemented within the robot development environment ArmarX. We evaluate the framework on the humanoid robot ARMAR-III in the context of two experiments: a demonstration of the real execution of a complex task in the kitchen environment on ARMAR-III and an experiment with untrained users in a simulation environment.