Abdel Rodríguez

Improving wet clutch engagement with reinforcement learning

A common approach when applying reinforcement learning to address control problems is that of first learning a policy based on an approximated model of the plant, whose behavior can be quickly and safely explored in simulation; and then implementing the obtained policy to control the actual plant. Here we follow this approach to learn to engage a transmission clutch, with the aim of obtaining a rapid and smooth engagement, with a small torque loss. Using an approximated model of a wet clutch, which simulates a portion of the whole engagement, we first learn an open loop control signal, which is then transferred on the actual wet clutch, and improved by further learning with a different reward function, based on the actual torque loss observed.

Comparing distance measures with visual methods

The selection of the distance measure to separate the objects of the knowledge space is critical in many classification algorithms. In this paper, we analyze the distance measures reported in the literature for the problem of HIV prediction. We propose a new distance for HIV viral sequences, based on the mutations with regard to the HXB2 reference sequence. In a first step, we reduce data dimensionality in order to subsequently analyze the distance measures performance in terms of its ability to separate classes.

Multi-Classifier Based on Hard Instances- New Method for Prediction of Human Immunodeficiency Virus Drug Resistance

There are several classification problems, which are difficult to solve using a single classifier because of the complexity of the decision boundary. Whereas a wide variety of multiple classifier systems have been built with the purpose of improving the recognition process, there is no universal method performing the best. This paper provides a review of different multi-classifiers and some application of them. Also it is shown a novel model of combining classifiers and its application to predicting human immunodeficiency virus drug resistance from genotype. The proposal is based on the use of different classifier models. It clusters the dataset considering the performance of the base classifiers. The system learns how to decide from the groups, by using a meta-classifier, which are the best classifiers for a given pattern. The proposed model is compared with well-known classifier ensembles and individual classifiers as well resulting the novel model in similar or even better performance.

A Reinforcement Learning Approach for Interdomain Routing with Link Prices

In today’s Internet, the commercial aspects of routing are gaining importance. Current technology allows Internet Service Providers (ISPs) to renegotiate contracts online to maximize profits. Changing link prices will influence interdomain routing policies that are now driven by monetary aspects as well as global resource and performance optimization. In this article, we consider an interdomain routing game in which the ISP’s action is to set the price for its transit links. Assuming a cheapest path routing scheme, the optimal action is the price setting that yields the highest utility (i.e., profit) and depends both on the network load and the actions of other ISPs. We adapt a continuous and a discrete action learning automaton (LA) to operate in this framework as a tool that can be used by ISP operators to learn optimal price setting. In our model, agents representing different ISPs learn only on the basis of local information and do not need any central coordination or sensitive information exchange. Simulation results show that a single ISP employing LAs is able to learn the optimal price in a stationary environment. By introducing a selective exploration rule, LAs are also able to operate in nonstationary environments. When two ISPs employ LAs, we show that they converge to stable and fair equilibrium strategies.

Model-free learning of wire winding control

In this paper we introduce a reinforcement learning approach to optimize the wire profile generated by an automated wire winding machine. The wire winder spools wire onto large bobbins, while trying to maintain an even wire profile across the bobbin. Uneven profiles that contain bumps or gaps (i.e. areas with too much or too little wire) lead to snagged or breaking wires when the bobbin is unwound. By setting the turning points of the traversal system which distributes the wire over a spinning bobbin, a controller can influence the amount of wire spooled on the edges of the bobbin. The behavior of the wire, however, is highly non-deterministic and difficult to model with sufficient accuracy, making the application of a model based controller technique very difficult. This fact makes reinforcement learning a promising approach to apply here, as this technique can learn optimal policies relying only on interactions with the plant. We apply a learning algorithm called continuous reinforcement learning automata and empirically demonstrate that this technique can successfully optimize the wire profile, even on rounded bobbins that require continuous adaptation of the turning point.

A reinforcement learning approach to coordinate exploration with limited communication in continuous action games

Learning automata are reinforcement learners belonging to the class of policy iterators. They have already been shown to exhibit nice convergence properties in a wide range of discrete action game settings. Recently, a new formulation for a continuous action reinforcement learning automata (CARLA) was proposed. In this paper, we study the behavior of these CARLA in continuous action games and propose a novel method for coordinated exploration of the joint-action space. Our method allows a team of independent learners, using CARLA, to find the optimal joint action in common interest settings. We first show that independent agents using CARLA will converge to a local optimum of the continuous action game. We then introduce a method for coordinated exploration which allows the team of agents to find the global optimum of the game. We validate our approach in a number of experiments.

Bidirectional Recurrent Neural Networks for Biological Sequences Prediction

The aim of this paper is to analyze the potentialities of Bidirectional Recurrent Neural Networks in classification problems. Different functions are proposed to merge the network outputs into one single classification decision. In order to analyze when these networks could be useful; artificial datasets were constructed to compare their performance against well-known classification methods in different situations, such as complex and simple decision boundaries, and related and independent features. The advantage of this neural network in classification problems with complicated decision boundaries and feature relations was proved statistically. Finally, better results using this network topology in the prediction of HIV drug resistance were also obtained.