Javier de Lope

Orthogonal variant moments features in image analysis

Moments are statistical measures used to obtain relevant information about a certain object under study (e.g., signals, images or waveforms), e.g., to describe the shape of an object to be recognized by a pattern recognition system. Invariant moments (e.g., the Hu invariant set) are a special kind of these statistical measures designed to remain constant after some transformations, such as object rotation, scaling, translation, or image illumination changes, in order to, e.g., improve the reliability of a pattern recognition system. The classical moment invariants methodology is based on the determination of a set of transformations (or perturbations) for which the system must remain unaltered. Although very well established, the classical moment invariants theory has been mainly used for processing single static images (i.e. snapshots) and the use of image moments to analyze images sequences or video, from a dynamic point of view, has not been sufficiently explored and is a subject of much interest nowadays. In this paper, we propose the use of variant moments as an alternative to the classical approach. This approach presents clear differences compared to the classical moment invariants approach, that in specific domains have important advantages. The difference between the classical invariant and the proposed variant approach is mainly (but not solely) conceptual: invariants are sensitive to any image change or perturbation for which they are not invariant, so any unexpected perturbation will affect the measurements (i.e. is subject to uncertainty); on the contrary, a variant moment is designed to be sensitive to a specific perturbation, i.e., to measure a transformation, not to be invariant to it, and thus if the specific perturbation occurs it will be measured; hence any unexpected disturbance will not affect the objective of the measurement confronting thus uncertainty. Furthermore, given the fact that the proposed variant moments are orthogonal (i.e. uncorrelated) it is possible to considerably reduce the total inherent uncertainty. The presented approach has been applied to interesting open problems in computer vision such as shape analysis, image segmentation, tracking object deformations and object motion tracking, obtaining encouraging results and proving the effectiveness of the proposed approach.

Adaptation, anticipation and rationality in natural and artificial systems: computational paradigms mimicking nature

Intelligence, Rationality, Learning, Anticipation and Adaptation are terms that have been and still remain in the central stage of computer science. These terms delimit their specific areas of study; nevertheless, they are so interrelated that studying them separately is an endeavor that seems little promising. In this paper, a model of study about the phenomena of Adaptation, Anticipation and Rationality as nature-inspired computational paradigms mimicking nature is proposed by means of a division, which is oriented, towards the discrimination of these terms, from the point of view of the complexity exhibited in the behavior of the systems, where these phenomena come at play. For this purpose a series of fundamental principles and hypothesis are proposed as well as some experimental results that corroborate them.

A method to learn the inverse kinematics of multi-link robots by evolving neuro-controllers

A general method to learn the inverse kinematic of multi-link robots by means of neuro-controllers is presented. We can find analytical solutions for the most used and well-known robots in the literature. However, these solutions are specific to a particular robot configuration and are not generally applicable to other robot morphologies. The proposed method is general in the sense that it is independent of the robot morphology. The method is based on the evolutionary computation paradigm and works obtaining incrementally better neuro-controllers. Furthermore, the proposed method solves some specific issues in robotic neuro-controller learning: it avoids any neural network learning algorithm which relies on the classical supervised input-target learning scheme and hence it lets to obtain neuro-controllers without providing targets. It can converge beyond local optimal solutions, which is one of the main drawbacks of some neural network training algorithms based on gradient descent when applied to highly redundant robot morphologies. Furthermore, using learning algorithms such as the neuro-evolution of augmenting topologies it is also possible to learn the neural network topology which is a common source of empirical testing in neuro-controllers design. Finally, experimental results are provided when applying the method to two multi-link robot learning tasks and a comparison between structural and parametric evolutionary strategies on neuro-controllers is shown.

Robust high performance reinforcement learning through weighted k-nearest neighbors

The aim of this paper is to present (jointly) a series of robust high performance (award winning) implementations of reinforcement learning algorithms based on temporal-difference learning and weighted k- nearest neighbors for linear function approximation. These algorithms, named kNN@?TD(@l) methods, where rigorously tested at the Second and Third Annual Reinforcement Learning Competitions (RLC2008 and RCL2009) held in Helsinki and Montreal respectively, where the kNN@?TD(@l) method (JAMH team) won in the PolyAthlon 2008 domain, obtained the second place in 2009 and also the second place in the Mountain-Car 2008 domain showing that it is one of the state of the art general purpose reinforcement learning implementations. These algorithms are able to learn quickly, to generalize properly over continuous state spaces and also to be robust to a high degree of environmental noise. Furthermore, we describe a derivation of kNN@?TD(@l) algorithm for problems where the use of continuous actions have clear advantages over the use of fine grained discrete actions: the Ex reinforcement learning algorithm.

Response threshold models and stochastic learning automata for self-coordination of heterogeneous multi-task distribution in multi-robot systems

This paper focuses on the general problem of coordinating multiple robots. More specifically, it addresses the self-selection of heterogeneous specialized tasks by autonomous robots. In this paper we focus on a specifically distributed or decentralized approach as we are particularly interested in a decentralized solution where the robots themselves autonomously and in an individual manner, are responsible for selecting a particular task so that all the existing tasks are optimally distributed and executed. In this regard, we have established an experimental scenario to solve the corresponding multi-task distribution problem and we propose a solution using two different approaches by applying Response Threshold Models as well as Learning Automata-based probabilistic algorithms. We have evaluated the robustness of the algorithms, perturbing the number of pending loads to simulate the robot’s error in estimating the real number of pending tasks and also the dynamic generation of loads through time. The paper ends with a critical discussion of experimental results.

Solving the Inverse Kinematics in Humanoid Robots: A Neural Approach

In this paper a method for solving the inverse kinematics of an humanoid robot based on artificial neural networks is presented. The input of the network is the desired positions and orientations of one foot with respect to the other foot. The output is the joint coordinates that make it possible to reach the goal configuration of the robot leg. To get a good set of sample data to train the neural network the direct kinematics of the robot needs to be developed, so to formulate the relationship between the joint variables and the position and orientation of the robot. Once this goal has been achieved, we need to establish the criteria we are going to use to choose from the range of possible joint configurations that fit with a particular foot position of the robot. These criteria will be used to filter all the possible configurations and retain the ones that make the robot configurations more stable in the training set.

Fusion of probabilistic knowledge-based classification rules and learning automata for automatic recognition of digital images

In this paper, the fusion of probabilistic knowledge-based classification rules and learning automata theory is proposed and as a result we present a set of probabilistic classification rules with self-learning capability. The probabilities of the classification rules change dynamically guided by a supervised reinforcement process aimed at obtaining an optimum classification accuracy. This novel classifier is applied to the automatic recognition of digital images corresponding to visual landmarks for the autonomous navigation of an unmanned aerial vehicle (UAV) developed by the authors. The classification accuracy of the proposed classifier and its comparison with well-established pattern recognition methods is finally reported.

The kNN-TD Reinforcement Learning Algorithm

A reinforcement learning algorithm called k NN-TD is introduced. This algorithm has been developed using the classical formulation of temporal difference methods and a k -nearest neighbors scheme as its expectations memory. By means of this kind of memory the algorithm is able to generalize properly over continuous state spaces and also take benefits from collective action selection and learning processes. Furthermore, with the addition of probability traces, we obtain the k NN-TD(*** ) algorithm which exhibits a state of the art performance. Finally the proposed algorithm has been tested on a series of well known reinforcement learning problems and also at the Second Annual RL Competition with excellent results.A reinforcement learning algorithm called kNN-TD is introduced. This algorithm has been developed using the classical formulation of temporal difference methods and a k-nearest neighbors scheme as its expectations memory. By means of this kind of memory the algorithm is able to generalize properly over continuous state spaces and also take benefits from collective action selection and learning processes. Furthermore, with the addition of probability traces, we obtain the kNN-TD(λ) algorithm which exhibits a state of the art performance. Finally the proposed algorithm has been tested on a series of well known reinforcement learning problems and also at the Second Annual RL Competition with excellent results.

Ex〈α〉: An effective algorithm for continuous actions Reinforcement Learning problems

In this paper the Ex(α) Reinforcement Learning algorithm is presented. This algorithm is designed to deal with problems where the use of continuous actions have clear advantages over the use of fine grained discrete actions. This new algorithm is derived from a baseline discrete actions algorithm implemented within a kind of fc-nearest neighbors approach in order to produce a probabilistic representation of the input signal to construct robust state descriptions based on a collection (knn) of receptive field units and a probability distribution vector p(knn) over the knn collection. The baseline continuous-space-discrete-actions fcNN-TD(A) algorithm introduces probability traces as the natural adaptation of eligibility traces in the probabilistic context. Later the Ex(α)(λ) algorithm is described as an extension of the baseline algorithms. Finally experimental results are presented for two (not easy) problems such as the Cart-Pole and Helicopter Hovering.

Integration of reactive utilitarian navigation and topological modeling

This chapter describes a hybrid autonomous navigation system for mobile robots. The control architecture proposed is highly modular and is based on the concept of behavior, which is a generalization of the usual reactive interpretation of this term. The proposed navigation system involves a straightforward integration of reactive and deliberative modules, enabling global, model-based navigation and local, adaptive navigation. At the local navigation level, we introduce the concept of utilitarian navigation, which models low-level robot navigation as a functional optimization process. Thanks to this innovative perspective, we have been able to implement low-level tasks, like collision avoidance and sensory source search and evasion, which have been integrated into the hybrid navigation system. At the global navigation level, two fundamental problems are considered: (1) map or model building and (2) route planning. Fuzzy Petri nets (FPN) are used to construct topological maps. A minimum cost algorithm of the FPN propagation has been implemented for route planning and execution. This chapter also discusses the experimental work carried out with realistic simulations, as well as with a holonomic prototype built by the authors and a NOMAD-200 mobile platform.