Darío Maravall

Hybrid Fuzzy Control of the Inverted Pendulum via Vertical Forces

In this article, we look at the excellent effect of vertical force as regards the stabilization of the inverted pendulum (IP) and demonstrate how the fuzzy control design methodology can be used to construct a hybrid fuzzy control system that incorporates PD control into a Takagi–Sugeno fuzzy control structure for stabilizing the IP via a vertical force. By gaining an intuitive understanding of the dynamics of the IP, the IP state space is fuzzily divided into six regions. In each region, a PD controller is designed to satisfy the stability conditions obtained by Lyapunovs direct and indirect methods. It shows that the proposed hybrid fuzzy control scheme provides a more flexible and intuitive way to stabilize the IP via a vertical force.

Adaptation, anticipation and rationality in natural and artificial systems: computational paradigms mimicking nature

Intelligence, Rationality, Learning, Anticipation and Adaptation are terms that have been and still remain in the central stage of computer science. These terms delimit their specific areas of study; nevertheless, they are so interrelated that studying them separately is an endeavor that seems little promising. In this paper, a model of study about the phenomena of Adaptation, Anticipation and Rationality as nature-inspired computational paradigms mimicking nature is proposed by means of a division, which is oriented, towards the discrimination of these terms, from the point of view of the complexity exhibited in the behavior of the systems, where these phenomena come at play. For this purpose a series of fundamental principles and hypothesis are proposed as well as some experimental results that corroborate them.

Robust high performance reinforcement learning through weighted k-nearest neighbors

The aim of this paper is to present (jointly) a series of robust high performance (award winning) implementations of reinforcement learning algorithms based on temporal-difference learning and weighted k- nearest neighbors for linear function approximation. These algorithms, named kNN@?TD(@l) methods, where rigorously tested at the Second and Third Annual Reinforcement Learning Competitions (RLC2008 and RCL2009) held in Helsinki and Montreal respectively, where the kNN@?TD(@l) method (JAMH team) won in the PolyAthlon 2008 domain, obtained the second place in 2009 and also the second place in the Mountain-Car 2008 domain showing that it is one of the state of the art general purpose reinforcement learning implementations. These algorithms are able to learn quickly, to generalize properly over continuous state spaces and also to be robust to a high degree of environmental noise. Furthermore, we describe a derivation of kNN@?TD(@l) algorithm for problems where the use of continuous actions have clear advantages over the use of fine grained discrete actions: the Ex reinforcement learning algorithm.

Combination of model-based and reactive methods in autonomous navigation

The paper presents three contributions to autonomous navigation of mobile robots: (1) a purely reactive method based on the potential field theory enhanced with a novel procedure to avoid local minima; (2) a topological map building method based on the sensory gradient concept that combined with the reactive module constitutes a hybrid navigation system and (3) another navigation system that uses the robots behaviors to automatically build a complete navigation model of the environment. All these schemes have been implemented on a Nomad-200 platform and fully tested in indoors environments.

Response threshold models and stochastic learning automata for self-coordination of heterogeneous multi-task distribution in multi-robot systems

This paper focuses on the general problem of coordinating multiple robots. More specifically, it addresses the self-selection of heterogeneous specialized tasks by autonomous robots. In this paper we focus on a specifically distributed or decentralized approach as we are particularly interested in a decentralized solution where the robots themselves autonomously and in an individual manner, are responsible for selecting a particular task so that all the existing tasks are optimally distributed and executed. In this regard, we have established an experimental scenario to solve the corresponding multi-task distribution problem and we propose a solution using two different approaches by applying Response Threshold Models as well as Learning Automata-based probabilistic algorithms. We have evaluated the robustness of the algorithms, perturbing the number of pending loads to simulate the robot’s error in estimating the real number of pending tasks and also the dynamic generation of loads through time. The paper ends with a critical discussion of experimental results.

Solving the Inverse Kinematics in Humanoid Robots: A Neural Approach

In this paper a method for solving the inverse kinematics of an humanoid robot based on artificial neural networks is presented. The input of the network is the desired positions and orientations of one foot with respect to the other foot. The output is the joint coordinates that make it possible to reach the goal configuration of the robot leg. To get a good set of sample data to train the neural network the direct kinematics of the robot needs to be developed, so to formulate the relationship between the joint variables and the position and orientation of the robot. Once this goal has been achieved, we need to establish the criteria we are going to use to choose from the range of possible joint configurations that fit with a particular foot position of the robot. These criteria will be used to filter all the possible configurations and retain the ones that make the robot configurations more stable in the training set.

A novel generalization of the gray-scale histogram and its application to the automated visual measurement and inspection of wooden Pallets

We present a novel concept, the histogram of connected elements (HCE) which is a generalization of the usual gray-level histogram of digital images is introduced and its application to automatic visual measurement and inspection system, currently operating at several European inspection plants. The main objective of the system is to automate the inspection of used wooden pallets. The paper begins with a brief description of the system, including some comments on the electromechanical handling of the inspected objects and the illumination set-up. Then, the paper presents the segmentation method used to extract the pallet elements, as an initial step for pallet measurements and the detection of possible defects. This method consists of an initial threshold on the histogram based on a Bayesian statistical classifier, followed by an iterative, heuristic search of the optimum threshold of the histogram. Finally, the paper introduces the application of the histogram of connected elements to the detection of very thin cracks, one of the hardest problems involved in the visual inspection of used pallets. Experimental results are obtained and we present a comparative study with several well-known and thoroughly tested techniques for the segmentation of textured images, including two algorithms belonging to the adaptive Bayesian family of restoration and segmentation methods and a probabilistic relaxation process.

Fusion of probabilistic knowledge-based classification rules and learning automata for automatic recognition of digital images

In this paper, the fusion of probabilistic knowledge-based classification rules and learning automata theory is proposed and as a result we present a set of probabilistic classification rules with self-learning capability. The probabilities of the classification rules change dynamically guided by a supervised reinforcement process aimed at obtaining an optimum classification accuracy. This novel classifier is applied to the automatic recognition of digital images corresponding to visual landmarks for the autonomous navigation of an unmanned aerial vehicle (UAV) developed by the authors. The classification accuracy of the proposed classifier and its comparison with well-established pattern recognition methods is finally reported.

The kNN-TD Reinforcement Learning Algorithm

A reinforcement learning algorithm called k NN-TD is introduced. This algorithm has been developed using the classical formulation of temporal difference methods and a k -nearest neighbors scheme as its expectations memory. By means of this kind of memory the algorithm is able to generalize properly over continuous state spaces and also take benefits from collective action selection and learning processes. Furthermore, with the addition of probability traces, we obtain the k NN-TD(*** ) algorithm which exhibits a state of the art performance. Finally the proposed algorithm has been tested on a series of well known reinforcement learning problems and also at the Second Annual RL Competition with excellent results.A reinforcement learning algorithm called kNN-TD is introduced. This algorithm has been developed using the classical formulation of temporal difference methods and a k-nearest neighbors scheme as its expectations memory. By means of this kind of memory the algorithm is able to generalize properly over continuous state spaces and also take benefits from collective action selection and learning processes. Furthermore, with the addition of probability traces, we obtain the kNN-TD(λ) algorithm which exhibits a state of the art performance. Finally the proposed algorithm has been tested on a series of well known reinforcement learning problems and also at the Second Annual RL Competition with excellent results.

Integration of reactive utilitarian navigation and topological modeling

This chapter describes a hybrid autonomous navigation system for mobile robots. The control architecture proposed is highly modular and is based on the concept of behavior, which is a generalization of the usual reactive interpretation of this term. The proposed navigation system involves a straightforward integration of reactive and deliberative modules, enabling global, model-based navigation and local, adaptive navigation. At the local navigation level, we introduce the concept of utilitarian navigation, which models low-level robot navigation as a functional optimization process. Thanks to this innovative perspective, we have been able to implement low-level tasks, like collision avoidance and sensory source search and evasion, which have been integrated into the hybrid navigation system. At the global navigation level, two fundamental problems are considered: (1) map or model building and (2) route planning. Fuzzy Petri nets (FPN) are used to construct topological maps. A minimum cost algorithm of the FPN propagation has been implemented for route planning and execution. This chapter also discusses the experimental work carried out with realistic simulations, as well as with a holonomic prototype built by the authors and a NOMAD-200 mobile platform.