Jose de Gea

Accelerating neuroevolutionary methods using a Kalman filter

In recent years, neuroevolutionary methods have shown great promise in solving learning tasks, especially in domains that are stochastic, partially observable, and noisy. In this paper, we show how the Kalman filter can be exploited (1) to efficiently find an optimal solution (i. e. reducing the number of evaluations needed to find the solution), (2) to find solutions that are robust against noise, and (3) to recover or reconstruct missing state variables, traditionally known as state estimation in control engineering community. Our algorithm has been tested on the double pole balancing without velocities benchmark, and has achieved significantly better results on this benchmark than the published results of other algorithms to date.

Bio-inspired control of an arm exoskeleton joint with active-compliant actuation system

This paper presents the methodology followed on the design of a multi-contact point haptic interface that uses a bio-inspired control approach and a novel actuation system. The combination of these components aims at creating a system that increases the operability of the target, and, at the same time, enables an intuitive and safe tele-operation of any complex robotic system of any given morphology. The novelty lies on the combination of a thoughtful kinematic structure driven by an active-compliant actuation system and a bio-inspired paradigm for its regulation. Due to the proposed actuation approach, the final system will achieve the condition of wearable system. On that final solution, each joint will be able to change its stiffness depending on the task to be executed, and on the anatomical features of each individual. Moreover, the system provides a variety of safety mechanisms at different levels to prevent causing any harm to the operator. In future, the system should allow the complete virtual immersion of the user within the working scenario.

A General Framework for Encoding and Evolving Neural Networks

In this paper we present a novel general framework for encoding and evolving networks called Common Genetic Encoding (CGE) that can be applied to both direct and indirect encoding methods. The encoding has important properties that makes it suitable for evolving neural networks: (1) It is completein that it is able to represent all types of valid phenotype networks. (2) It is closed, i. e. every valid genotype represents a valid phenotype. Similarly, the encoding is closed under genetic operatorssuch as structural mutation and crossover that act upon the genotype. Moreover, the encodings genotype can be seen as a composition of several subgenomes, which makes it to inherently support the evolution of modular networks in both direct and indirect encoding cases. To demonstrate our encoding, we present an experiment where direct encoding is used to learn the dynamic model of a two-link arm robot. We also provide an illustration of how the indirect-encoding features of CGE can be used in the area of artificial embryogeny.

Design and control of an intelligent dual-arm manipulator for fault-recovery in a production scenario

This paper describes the design and control methodology used for the development of a dual-arm manipulator as well as its deployment in a production scenario. Multi-modal and sensor-based manipulation strategies are used to guide the robot on its task to supervise and, when necessary, solve faulty situations in a production line. For that task the robot is equipped with two arms, aimed at providing the robot with total independence from the production line. In other words, no extra mechanical stoppers are mounted on the line to halt targeted objects, but the robot will employ both arms to (a) stop with one arm a carrier that holds an object to be inserted/replaced, and (b) use the second arm to handle such object. Besides, visual information from head and wrist-mounted cameras provide the robot with information such as the state of the production line, the unequivocal detection/recognition of the targeted objects, and the location of the target in order to guide the grasp.

Learning Walking Patterns for Kinematically Complex Robots Using Evolution Strategies

Manually developing walking patterns for kinematically complex robots can be a challenging and time-consuming task. In order to automate this design process, a learning system that generates, tests, and optimizes different walking patterns is needed, as well as the ability to accurately simulate a robot and its environment. In this work, we describe a learning system that uses the CMA-ES method from evolutionary computation to learn walking patterns for a complex legged robot. The robots limbs are controlled using parametrized distorted sine waves, and the evolutionary algorithm optimizes the parameters of these waveforms, testing the walking patterns in a physical simulation. The best solutions evolved by this system has been transferred to and tested on a real robot, and has resulted in a gait that is superior to those previously designed by a human designer.

On Applying Neuroevolutionary Methods to Complex Robotic Tasks

In this paper, we describe possible methods of solving two problems encountered in evolutionary robotics, while applying neuroevolutionary methods to evolve controllers for complex robotic tasks. The first problem is the large number of evaluations required to obtain a solution. We propose that this problem can be addressed by accelerating neuroevolutionary methods using a Kalman filter. The second problem is the difficulty of obtaining a desirable solution that results from the difficulty of defining an appropriate fitness function for a complex robotic task. The solution towards this problem is to apply the principles of behavior based systems to decompose the solution space into smaller subsolutions with lower number of intrinsic dimensions, and incrementally modify the fitness function. We present two case studies towards the solutions to the stated problems.

Modelling and Simulation of Robot Arm Interaction Forces Using Impedance Control

In this paper we present the implementation of a Cartesian impedance control method to regulate the interaction forces between a robotic arm and the environment. A complete description of the procedure to model and control both a two-link planar robot arm and its interaction with the environment is detailed and simulated using MATLAB/Simulink; from the generation of a mechanical model in SimMechanics (MATLAB), the description and tuning of a dynamic model-based controller to cancel-out the non-linearities present on the dynamic model of the robot, the modelling of an environment, and finally the control of the interaction forces making use of a Cartesian impedance control method. This type of control adjusts the dynamic behaviour of the robot manipulator when contacting the environment, basically controlling stiffness and damping of the interaction rather than the precise contact forces. Its implementation in the Cartesian Space permits future use of the results in an industrial robot, whose internal joint and torque controllers are commonly not accessible.

Control of Robot Interaction Forces Using Evolutionary Techniques

For a robot manipulator to interact safely and human-friendly in an unknown environment, it is necessary to include an interaction control method that reliably adapts the forces exerted on the environment in order to avoid damages both to the environment and to the manipulator itself. A force control method, or strictly speaking, a direct force control method, can be used on those applications where the maximum or the desired force to exert is known beforehand. In some industrial applications the objects to handle or work with are completely known as well as the precise moment on which these contacts are going to happen. In a more general scenario, such as one outside a well-defined robotic workcell or when an industrial robot is used in cooperation with a human, neither the objects nor the time when a contact is ocurring are known. In such a case, indirect force control methods find their niche. These methods do not seek to control maximum or desired force, but they try to make the manipulator compliant with the object being contacted. The major role in the control loop is given to the positioning but the interaction is also being controlled so as to ensure a safe and clear contact. In case contact interaction forces have exceeded the desired levels, the positioning accuracy will be diminished to account and take care of the (at this moment) most important task: the control of the forces. Impedance control (Hogan (1985)) is one of these indirect force control methods. Its aim is to control the dynamic behaviour of a robot manipulator when contacting the environment, not by controlling the exact contact forces but the properties of the contact, namely, controlling the stiffness and the damping of the interaction. Moreover, the steady-state force can be easily set to a desired maximum value. The main idea is that the impedance control system creates a virtual new impedance for the manipulator, which is being able to interact with the environment as if new mechanical elements had been included in the real manipulator. First industrial approaches were focused on controlling the force exerted on the environment by a direct force feedback loop. A state-of-the-art review of the 80s is provided in (Whitney (1987)) and the progress during the 90s is described in (Schutter et al. (1997)). In many in22

EANT+KALMAN: An Efficient Reinforcement Learning Method for Continuous State Partially Observable Domains

In this contribution we present an extension of a neuroevolutionary method called Evolutionary Acquisition of Neural Topologies (EANT) [11] that allows the evolution of solutions taking the form of a POMDP agent (Partially Observable Markov Decision Process) [8]. The solution we propose involves cascading a Kalman filter [10] (state estimator) and a feed-forward neural network. The extension (EANT+KALMAN) has been tested on the double pole balancing without velocity benchmark, achieving significantly better results than the to date published results of other algorithms.

Learning complex robot control using evolutionary behavior based systems

Evolving a monolithic solution for complex robotic problems is hard. One of the reasons for this is the difficulty of defining a global fitness function that leads to a solution with desired operating properties. The problem with a global fitness function is that it may not reward intermediate solutions that would ultimately lead to the desired operating properties. A possible way to solve such a problem is to decompose the solution space into smaller subsolutions with lower number of intrinsic dimensions. In this paper, we apply the design principles of behavior based systems to decompose a complex robot control task into subsolutions and show how to incrementally modify the fitness function that (1) results in desired operating properties as the subsolutions are learned, and (2) avoids the need to learn the coordination of behaviors separately. We demonstrate our method by learning to control a quadrocopter flying vehicle.