Mirko Wächter

Action sequence reproduction based on automatic segmentation and Object-Action Complexes

Teaching robots object manipulation skills is a complex task that involves multimodal perception and knowledge about processing the sensor data. In this paper, we show a concept for humanoid robots in household environments with a variety of related objects and actions. Following the paradigms of Programming by Demonstration (PbD), we provide a flexible approach that enables a robot to adaptively reproduce an action sequence demonstrated by a human. The obtained human motion data with involved objects is segmented into semantic conclusive sub-actions by the detection of relations between the objects and the human actor. Matching actions are chosen from a library of Object-Action Complexes (OACs) using the preconditions and effects of each sub-action. The resulting sequence of OACs is parameterized for the execution on a humanoid robot depending on the observed action sequence and on the state of the environment during execution. The feasibility of this approach is shown in an exemplary kitchen scenario, where the robot has to prepare a dough.

The robot software framework ArmarX

Abstract With ArmarX we introduce a robot programming environment that has been developed in order to ease the realization of higher level capabilities needed by complex robotic systems such as humanoid robots. ArmarX is built upon the idea that consistent disclosure of the system state strongly facilitates the development process of distributed robot applications. We show the applicability of ArmarX by introducing a robot architecture for a humanoid system and discuss essential aspects based on an exemplary pick and place task. With several tools that are provided by the ArmarX framework, such as graphical user interfaces (GUI) or statechart editors, the programmer is enabled to efficiently build and inspect component based robotics software systems.

Structural Bootstrapping—A Novel, Generative Mechanism for Faster and More Efficient Acquisition of Action-Knowledge

Humans, but also robots, learn to improve their behavior. Without existing knowledge, learning either needs to be explorative and, thus, slow or-to be more efficient-it needs to rely on supervision, which may not always be available. However, once some knowledge base exists an agent can make use of it to improve learning efficiency and speed. This happens for our children at the age of around three when they very quickly begin to assimilate new information by making guided guesses how this fits to their prior knowledge. This is a very efficient generative learning mechanism in the sense that the existing knowledge is generalized into as-yet unexplored, novel domains. So far generative learning has not been employed for robots and robot learning remains to be a slow and tedious process. The goal of the current study is to devise for the first time a general framework for a generative process that will improve learning and which can be applied at all different levels of the robots cognitive architecture. To this end, we introduce the concept of structural bootstrapping-borrowed and modified from child language acquisition-to define a probabilistic process that uses existing knowledge together with new observations to supplement our robots data-base with missing information about planning-, object-, as well as, action-relevant entities. In a kitchen scenario, we use the example of making batter by pouring and mixing two components and show that the agent can efficiently acquire new knowledge about planning operators, objects as well as required motor pattern for stirring by structural bootstrapping. Some benchmarks are shown, too, that demonstrate how structural bootstrapping improves performance.

Hierarchical segmentation of manipulation actions based on object relations and motion characteristics

Understanding human actions is an indispensable capability of humanoid robots which acquire task knowledge from human demonstration. Segmentation of such continuous demonstrations into meaningful segments reduces the complexity of understanding an observed task. In this paper, we propose a two-level hierarchical action segmentation approach which considers semantics of an action in addition to human motion characteristics. On the first level, a semantic segmentation is performed based on contact relations between human end-effectors, the scene, and between objects in the scene. On the second level, the semantic segments are further sub-divided based on a novel heuristic that incorporates the motion characteristics into the segmentation procedure. As input for the segmentation, we present an observation method for tracking the human as well as the objects and the environment. 6D pose trajectories of the humans hands and all objects are extracted in a precise and robust manner from data of a marker-based tracking system. We evaluated and compared our approach with a manual reference segmentation and well-known segmentation algorithms based on PCA and zero-velocity-crossings using 13 human demonstrations of daily activities.We show that significantly smaller segmentation errors are achieved with our approach while providing the necessary granularity for representing human demonstrations.

Multi-purpose natural language understanding linked to sensorimotor experience in humanoid robots

Humans have an amazing ability to bootstrap new knowledge. The concept of structural bootstrapping refers to mechanisms relying on prior knowledge, sensorimotor experience, and inference that can be implemented in robotic systems and employed to speed up learning and problem solving in new environments. In this context, the interplay between the symbolic encoding of the sensorimotor information, prior knowledge, planning, and natural language understanding plays a significant role. In this paper, we show how the symbolic descriptions of the world can be generated on the fly from the continuous robots memory. We also introduce a multi-purpose natural language understanding framework that processes human spoken utterances and generates planner goals as well as symbolic descriptions of the world and human actions. Both components were tested on the humanoid robot ARMAR-III in a scenario requiring planning and plan recognition based on human-robot communication.

Modulation of motor primitives using force feedback: Interaction with the environment and bimanual tasks

The framework of dynamic movement primitives allows the generation of discrete and periodic trajectories, which can be modulated in various aspects. We propose and evaluate a novel modulation approach that includes force feedback and thus allows physical interaction with objects and the environment. The proposed approach also enables the coupling of independently executed robotic trajectories, simplifying the execution of bimanual and tightly coupled cooperative tasks. We apply an iterative learning control algorithm to learn a coupling term, which is applied to the original trajectory in a feed-forward fashion. The coupling term modifies the trajectory in accordance to either the desired position or external force. The strengths of the approach are shown in bimanual or two-agent obstacle avoidance tasks, where no higher level cognitive reasoning or planning are required. Results of simulated and real-world experiments on the ARMAR-III humanoid robot in interaction and object lifting tasks, and on two KUKA LWR robots in a bimanual setting are presented.

The ArmarX Statechart Concept: Graphical Programing of Robot Behavior

Programming sophisticated robots such as service robots or humanoids is still a complex endeavor. Although programming robotic applications requires specialist knowledge, a robot software environment should support convenient development while maintaining full flexibility needed when realizing challenging robotics tasks. In addition, several desirable properties should be fulfilled, such as robustness, reusability of existing programs, and skill transfer between robots. In this work, we introduce the ArmarX statechart concept, which is used for describing control and data flow of robot programs. This event-driven statechart approach of ArmarX helps realizing important features such as increased robustness through distributed program execution, convenient programming through graphical user interfaces, and versatility by interweaving dynamic statechart structure with custom user-code. We show that using hierarchical and distributed statecharts increases reusability, allows skill transfer between robots, and hides complexity in robot programming by splitting robot behavior into control flow and functionality.

Transferring object grasping knowledge and skill across different robotic platforms

This study describes the transfer of object grasping skills between two different humanoid robots with different software frameworks. We realize such a knowledge and skill transfer between the humanoid robots iCub and ARMAR-III. These two robots have different kinematics and are programmed using different middlewares, YARP and ArmarX. We developed a bridge system to allow for the execution of grasping skills of ARMAR-III on iCub. As the embodiment differs, the known feasible grasps for the one robot are not always feasible for the other robot. We propose a reactive correction behavior to detect failure of a grasp during its execution, to correct it until it is successful, and thus adapt the known grasp definition to the new embodiment.

Workspace analysis for planning human-robot interaction tasks

We present an approach for determining suitable locations for human-robot interaction tasks. Therefore, we introduce the task specific Interaction Workspace as a representation of the workspace that can be accessed by both agents, i.e. the robot and the human. We show how the Interaction Workspace can be efficiently determined for a specific situation by making use of precomputed workspace representations of robot and human. By considering several quality measures related to dexterity and comfort, the Interaction Workspace provides valuable information about potential targets for human robot interaction (e.g. for object handover tasks). We evaluate the online performance of building appropriate data structures and show how the approach can be applied in a realistic hand-over use case with the humanoid robot ARMAR-III.

A skeleton-based approach to grasp known objects with a humanoid robot

This paper is about grasping known objects of arbitrary shape with a humanoid robot. We extend our previous work, where we presented a grasp planning method using an object representation based on the medial axis transform (MAT). The MAT describes an objects topological skeleton and contains information about local symmetry properties and thickness valuable for grasp planning. So far, our previous work was only conducted in simulation. The contribution of this paper is the transfer of our grasp planning method to the real world. We present grasping experiments with challenging arbitrarily shaped objects where we execute the grasps generated by our grasp planner on a real humanoid robot with a five-finger hand.