Nikolay Jetchev

Integrated motor control, planning, grasping and high-level reasoning in a blocks world using probabilistic inference

A new approach to planning and goal-directed behavior has recently been proposed using probabilistic inference in a graphical model that represents states, actions, constraints and goals of the future to infer appropriate actions and controls. The approach has led to new algorithms on the control and trajectory optimization level as well as for high-level rule-based planning in relational domains. In this paper we integrate these methods to a coherent control, trajectory optimization, and action planning architecture, using the principle of planning by inference across all levels of abstractions. Our scenario is a real blocks world: using a 14DoF Schunk arm and hand with tactile sensors and a stereo camera, the goal is to manipulate a set of objects on the table in a goal-oriented way. For high-level reasoning, we learn relational rule-based models from experience in simulation.

Trajectory prediction: learning to map situations to robot trajectories

Trajectory planning and optimization is a fundamental problem in articulated robotics. Algorithms used typically for this problem compute optimal trajectories from scratch in a new situation. In effect, extensive data is accumulated containing situations together with the respective optimized trajectories - but this data is in practice hardly exploited. The aim of this paper is to learn from this data. Given a new situation we want to predict a suitable trajectory which only needs minor refinement by a conventional optimizer. Our approach has two essential ingredients. First, to generalize from previous situations to new ones we need an appropriate situation descriptor - we propose a sparse feature selection approach to find such well-generalizing features of situations. Second, the transfer of previously optimized trajectories to a new situation should not be made in joint angle space - we propose a more efficient task space transfer of old trajectories to new situations. Experiments on a simulated humanoid reaching problem show that we can predict reasonable motion prototypes in new situations for which the refinement is much faster than an optimization from scratch.

Fast motion planning from experience: trajectory prediction for speeding up movement generation

Trajectory planning and optimization is a fundamental problem in articulated robotics. Algorithms used typically for this problem compute optimal trajectories from scratch in a new situation. In effect, extensive data is accumulated containing situations together with the respective optimized trajectories—but this data is in practice hardly exploited. This article describes a novel method to learn from such data and speed up motion generation, a method we denote tajectory pediction. The main idea is to use demonstrated optimal motions to quickly predict appropriate trajectories for novel situations. These can be used to initialize and thereby drastically speed-up subsequent optimization of robotic movements. Our approach has two essential ingredients. First, to generalize from previous situations to new ones we need a situation descriptor—we construct features for such descriptors and use a sparse regularized feature selection approach to improve generalization. Second, the transfer of previously optimized trajectories to a new situation should not be made in joint angle space—we propose a more efficient task space transfer. We present extensive results in simulation to illustrate the benefits of the new method, and demonstrate it also with real robot hardware. Our experiments in diverse tasks show that we can predict good motion trajectories in new situations for which the refinement is much faster than an optimization from scratch.

Optimization of fluent approach and grasp motions

Generating a fluent motion of approaching, grasping and lifting an object comprises a number of problems which are typically tackled separately. Some existing research specializes on the optimization of the final grasp posture based on force closure criteria neglecting the motion necessary to approach this grasp. Other research specializes on motion optimization including collision avoidance criteria, but typically not considering the subsequent grasp as part of the optimization problem. In this paper we aim to combine existing techniques for grasp optimization, trajectory optimization, and attractor-based movement representation, into a comprehensive framework that allows us to efficiently compute a fluent approach and grasping motion. The feasibility of the proposed approach is shown in simulation studies and experiments with a humanoid robot.

Trajectory prediction in cluttered voxel environments

Trajectory planning and optimization is a fundamental problem in articulated robotics. It is often viewed as a two phase problem of initial feasible path planning around obstacles and subsequent optimization of a trajectory satisfying dynamical constraints. There are many methods that can generate good movements when given enough time, but planning for high-dimensional robot configuration spaces in realistic environments with many objects in real time remains challenging. This work presents a novel way for faster movement planning in such environments by predicting good path initializations. We build on our previous work on trajectory prediction by adapting it to environments modeled with voxel grids and defining a frame invariant prototype trajectory space. The constructed representations can generalize to a wide range of situations, allowing to predict good movement trajectories and speed up convergence of robot motion planning. An empirical comparison of the effect on planning movements with a combination of different trajectory initializations and local planners is presented and tested on a Schunk arm manipulation platform with laser sensors in simulation and hardware.

Discovering relevant task spaces using inverse feedback control

Learning complex skills by repeating and generalizing expert behavior is a fundamental problem in robotics. However, the usual approaches do not answer the question of what are appropriate representations to generate motion for a specific task. Since it is time-consuming for a human expert to manually design the motion control representation for a task, we propose to uncover such structure from data-observed motion trajectories. Inspired by Inverse Optimal Control, we present a novel method to learn a latent value function, imitate and generalize demonstrated behavior, and discover a task relevant motion representation. We test our method, called Task Space Retrieval Using Inverse Feedback Control (TRIC), on several challenging high-dimensional tasks. TRIC learns the important control dimensions for the tasks from a few example movements and is able to robustly generalize to new situations.

Optimizing motion primitives to make symbolic models more predictive

Solving complex robot manipulation tasks requires to combine motion generation on the geometric level with planning on a symbolic level. On both levels robotics research has developed a variety of mature methodologies, including geometric motion planning and motion primitive learning on the motor level as well as logic reasoning and relational Reinforcement Learning methods on the symbolic level. However, their robust integration remains a great challenge. In this paper we approach one aspect of this integration by optimizing the motion primitives on the geometric level to be as consistent as possible with their symbolic predictions. The so optimized motion primitives increase the probability of a “successful” motion-meaning that the symbolic prediction was indeed achieved. Conversely, using these optimized motion primitives to collect new data about the effects of actions the learnt symbolic rules becomes more predictive and deterministic.

Kognitive Robotik — Herausforderungen an unser Verständnis natürlicher Umgebungen

Zusammenfassung Wir diskutieren einen Ansatz, der kognitive Robotik als die Erweiterung der Methoden der Robotik — insb. Lernen, Planen und Regelung — auf „äußere Freiheitsgrade“ versteht. Damit verschiebt sich der Fokus: Weg von Gelenkwinkeln, Vektorräumen und Gaußschen Verteilungen, hin zu den Objekten und der Struktur der Umwelt. Letztere können wir nur schwer formalisieren und in geeignete Repräsentationen und Priors übersetzen, mit denen effizientes Lernen und Planen möglich wird. Es wird deutlich, welche theoretischen Probleme sich hinter dem Ziel automomer Systeme verbergen, die durch intelligente Exploration und Verallgemeinerung ihre Umwelt zu verstehen lernen und die gelernten Modelle zur Handlungsplanung nutzen. Die momentan diskutierte Integration von Logik, Geometrie und Wahrscheinlichkeiten — und damit die Überbrückung der klassischen Disziplinbarrieren zwischen Robotik, Künstlicher Intelligenz und statistischer Lerntheorie — ist eine der zwangsläufigen Herausforderungen der kognitiven Robotik. In diesem Kontext skizzieren wir eigene Beiträge zum relationalen Reinforcement-Lernen, zur Exploration und dem Symbol-Lernen. Abstract We discuss that “cognitive robotics” implies the extension of classical robotics methods — esp. planning, control and learning — to external degrees of freedom (DoFs). External refers to the articulated and manipulable DoFs of the environment and the objects therein. Coping with these DoFs requires to go beyond vector spaces and Gaussian distributions and instead address the complex structure of natural environments, which is hard to formalize and translate to appropriate representations and priors for efficient learning. With this discussion we aim to highlight the theoretical challenges behind the goal of robots that autonomously explore their environment and learn to manipulated external DoFs. The integration of logic, geometry and probabilities is one of these challenges. In this context we briefly sketch own work on relational reinforcement learning, exploration and symbol learning.