João Carlos Gama Martins Macedo

Using Cost-regularized Kernel Regression with a high number of samples

The CrKR (Cost-regularized Kernel Regression) algorithm allows to optimize a function that dictates the meta-parameters to be used for a motor task according to a state variable. Particularly, it stood out at optimizing the meta-parameters of Dynamic Motion Primitives (DMPs) used in striking movements in robot table tennis. However, the high computational complexity of the algorithm limits the number of training samples that can be used before running out of computational resources to keep its execution. A method that minimizes this issue is here proposed, based on the idea that we can split the execution of the algorithm in several runs with a smaller number of rollouts, while transferring the learned state to meta-parameters function between successive runs. This solution allows circumventing the original algorithm limitations, making it possible to use with a high number of training samples without having to deal with an indefinitely increasing execution time per iteration. We compare the effectiveness and execution time of the algorithm before and after applying this transformation and verify that the first is almost unaffected while the latter decreases substantially.

Toward a flexible framework for learning: F3L

Abstract For stable and adaptable movement generation, it is required that movement primitives are generalizable in order to be executed in different contexts and for distinct instances of the same task. Therefore, a motion primitive must adapt policy parameters according to environment variables, instead of repetitively executing the exact same motor commands when placed into a specific context. Another important aspect is how the learning process of motion primitives is guided. Some movements are too complex to allow its primitives to be learned all at once, i.e, learning all their intricacies without a properly structured approach. In order to simplify this process, a framework by the name of Flexible Framework for Learning (F3L) , that allows storing, combining and mutual and partial learning of parameterized policies, is presented. It provides the means, for instance, to build a movement primitive library or to perform offline gradual learning of a specific motor skill. Additionally, F3L is used to teach the controller of a biped robot (DARwIn-OP) to adapt the parameters of CPG-based locomotion according to the slope angles of the underlying surface. The introduction of several novel feedback mechanisms is also suggested. The results achieved are gratifying, promising and attest the fact that F3L can provide an effective and useful tool to structure the learning process of complex tasks. The direction correction feedback mechanism, in addition to keeping the robot walking in a straight path, has shown the ability to control walking direction bypassing internal CPG dynamics.