Danilo Bruno

A task-parameterized probabilistic model with minimal intervention control

We present a task-parameterized probabilistic model encoding movements in the form of virtual spring-damper systems acting in multiple frames of reference. Each candidate coordinate system observes a set of demonstrations from its own perspective, by extracting an attractor path whose variations depend on the relevance of the frame at each step of the task. This information is exploited to generate new attractor paths in new situations (new position and orientation of the frames), with the predicted covariances used to estimate the varying stiffness and damping of the spring-damper systems, resulting in a minimal intervention control strategy. The approach is tested with a 7-DOFs Barrett WAM manipulator whose movement and impedance behavior need to be modulated in regard to the position and orientation of two external objects varying during demonstration and reproduction.

Learning optimal controllers in human-robot cooperative transportation tasks with position and force constraints

Human-robot collaboration seeks to have humans and robots closely interacting in everyday situations. For some tasks, physical contact between the user and the robot may occur, originating significant challenges at safety, cognition, perception and control levels, among others. This paper focuses on robot motion adaptation to parameters of a collaborative task, extraction of the desired robot behavior, and variable impedance control for human-safe interaction. We propose to teach a robot cooperative behaviors from demonstrations, which are probabilistically encoded by a task-parametrized formulation of a Gaussian mixture model. Such encoding is later used for specifying both the desired state of the robot, and an optimal feedback control law that exploits the variability in position, velocity and force spaces observed during the demonstrations. The whole framework allows the robot to modify its movements as a function of parameters of the task, while showing different impedance behaviors. Tests were successfully carried out in a scenario where a 7 DOF backdrivable manipulator learns to cooperate with a human to transport an object.

Human-robot skills transfer interfaces for a flexible surgical robot

In minimally invasive surgery, tools go through narrow openings and manipulate soft organs to perform surgical tasks. There are limitations in current robot-assisted surgical systems due to the rigidity of robot tools. The aim of the STIFF-FLOP European project is to develop a soft robotic arm to perform surgical tasks. The flexibility of the robot allows the surgeon to move within organs to reach remote areas inside the body and perform challenging procedures in laparoscopy. This article addresses the problem of designing learning interfaces enabling the transfer of skills from human demonstration. Robot programming by demonstration encompasses a wide range of learning strategies, from simple mimicking of the demonstrators actions to the higher level imitation of the underlying intent extracted from the demonstrations. By focusing on this last form, we study the problem of extracting an objective function explaining the demonstrations from an over-specified set of candidate reward functions, and using this information for self-refinement of the skill. In contrast to inverse reinforcement learning strategies that attempt to explain the observations with reward functions defined for the entire task (or a set of pre-defined reward profiles active for different parts of the task), the proposed approach is based on context-dependent reward-weighted learning, where the robot can learn the relevance of candidate objective functions with respect to the current phase of the task or encountered situation. The robot then exploits this information for skills refinement in the policy parameters space. The proposed approach is tested in simulation with a cutting task performed by the STIFF-FLOP flexible robot, using kinesthetic demonstrations from a Barrett WAM manipulator.

Skills transfer across dissimilar robots by learning context-dependent rewards

Robot programming by demonstration encompasses a wide range of learning strategies, from simple mimicking of the demonstrators actions to the higher level extraction of the underlying intent. By focusing on this last form, we study the problem of extracting the reward function explaining the demonstrations from a set of candidate reward functions, and using this information for self-refinement of the skill. This definition of the problem has links with inverse reinforcement learning problems in which the robot autonomously extracts an optimal reward function that defines the goal of the task. By relying on Gaussian mixture models, the proposed approach learns how the different candidate reward functions are combined, and in which contexts or phases of the task they are relevant for explaining the users demonstrations. The extracted reward profile is then exploited to improve the skill with a self-refinement approach based on expectation-maximization, allowing the imitator to reach a skill level that goes beyond the demonstrations. The approach can be used to reproduce a skill in different ways or to transfer tasks across robots of different structures. The proposed approach is tested in simulation with a new type of continuum robot (STIFF-FLOP), using kinesthetic demonstrations from a Barrett WAM manipulator.

Learning adaptive movements from demonstration and self-guided exploration

The combination of imitation and exploration strategies is used in this paper to transfer sensory-motor skills to robotic platforms. The aim is to be able to learn very different tasks with good generalization capabilities and starting from a few demonstrations. This goal is achieved by learning a task-parameterized model from demonstrations where a teacher shows the task corresponding to different possible values of preassigned parameters. In this manner, new reproductions can be generated for new situations by assigning new values to the parameters, thus achieving very precise generalization capabilities. In this paper we propose a novel algorithm that is able to learn the model together with its dependence from the task-parameters, without specifying a predefined relationship or structure. The algorithm is able to learn the model starting from a few demonstrations by applying an exploration strategy that refines the learnt model autonomously. The algorithm is tested on a reaching task performed with a Barrett WAM manipulator.

Learning autonomous behaviours for the body of a flexible surgical robot

This paper presents a novel strategy to learn a positional controller for the body of a flexible surgical manipulator used for minimally invasive surgery. The manipulator is developed within the STIFF-FLOP European project and is targeted for a laparoscopic use in remote areas of the abdominal region that are not easily accessible by means of currently available rigid tools. While the surgeon controls the end-effector during the task, the flexible body of the manipulator needs to be displaced to enter inside constrained spaces by efficiently exploiting its flexibility, without touching vital organs and structures. The proposed algorithm exploits the instruments of machine learning within the programming by demonstrations paradigm to produce a statistical model of the natural movements of the surgeon during the task. The gathered information is then reused to determine a controller in the null space of the robot that does not interfere with the surgeon task and displaces the robot body within the available space in a fully automated manner.

Null space redundancy learning for a flexible surgical robot

A new challenge for surgical robotics is placed in the use of flexible manipulators, to perform procedures that are impossible for currently available rigid robots. Since the surgeon only controls the end-effector of the manipulator, new control strategies need to be developed to correctly move its flexible body without damaging the surrounding environment. This paper shows how a positional controller for a new surgical robot (STIFF-FLOP) can be learnt from the demonstrations given by an expert user. The proposed algorithm exploits the variability of the task to comply with the constraints only when needed, by implementing a minimal intervention principle control strategy. The results are applied to scenarios involving movements inside a constrained environment and disturbance rejection.

Learning the Stiffness of a Continuous Soft Manipulator from Multiple Demonstrations

Continuous soft robots are becoming more and more widespread in applications, due to their increased safety and flexibility in critical applications. The possibility of having soft robots that are able to change their stiffness in selected parts can help in situations where higher forces need to be applied. This paper describes a theoretical framework for learning the desired stiffness characteristics of the robot from multiple demonstrations. The framework is based on a statistical mathematical model for encoding the motion of a continuous manipulator, coupled with an optimal control strategy for learning the best impedance parameters of the manipulator.