H José Antonio Martín

Orthogonal variant moments features in image analysis

Moments are statistical measures used to obtain relevant information about a certain object under study (e.g., signals, images or waveforms), e.g., to describe the shape of an object to be recognized by a pattern recognition system. Invariant moments (e.g., the Hu invariant set) are a special kind of these statistical measures designed to remain constant after some transformations, such as object rotation, scaling, translation, or image illumination changes, in order to, e.g., improve the reliability of a pattern recognition system. The classical moment invariants methodology is based on the determination of a set of transformations (or perturbations) for which the system must remain unaltered. Although very well established, the classical moment invariants theory has been mainly used for processing single static images (i.e. snapshots) and the use of image moments to analyze images sequences or video, from a dynamic point of view, has not been sufficiently explored and is a subject of much interest nowadays. In this paper, we propose the use of variant moments as an alternative to the classical approach. This approach presents clear differences compared to the classical moment invariants approach, that in specific domains have important advantages. The difference between the classical invariant and the proposed variant approach is mainly (but not solely) conceptual: invariants are sensitive to any image change or perturbation for which they are not invariant, so any unexpected perturbation will affect the measurements (i.e. is subject to uncertainty); on the contrary, a variant moment is designed to be sensitive to a specific perturbation, i.e., to measure a transformation, not to be invariant to it, and thus if the specific perturbation occurs it will be measured; hence any unexpected disturbance will not affect the objective of the measurement confronting thus uncertainty. Furthermore, given the fact that the proposed variant moments are orthogonal (i.e. uncorrelated) it is possible to considerably reduce the total inherent uncertainty. The presented approach has been applied to interesting open problems in computer vision such as shape analysis, image segmentation, tracking object deformations and object motion tracking, obtaining encouraging results and proving the effectiveness of the proposed approach.

Adaptation, anticipation and rationality in natural and artificial systems: computational paradigms mimicking nature

Intelligence, Rationality, Learning, Anticipation and Adaptation are terms that have been and still remain in the central stage of computer science. These terms delimit their specific areas of study; nevertheless, they are so interrelated that studying them separately is an endeavor that seems little promising. In this paper, a model of study about the phenomena of Adaptation, Anticipation and Rationality as nature-inspired computational paradigms mimicking nature is proposed by means of a division, which is oriented, towards the discrimination of these terms, from the point of view of the complexity exhibited in the behavior of the systems, where these phenomena come at play. For this purpose a series of fundamental principles and hypothesis are proposed as well as some experimental results that corroborate them.

FPGA-Based Multimodal Embedded Sensor System Integrating Low- and Mid-Level Vision

Motion estimation is a low-level vision task that is especially relevant due to its wide range of applications in the real world. Many of the best motion estimation algorithms include some of the features that are found in mammalians, which would demand huge computational resources and therefore are not usually available in real-time. In this paper we present a novel bioinspired sensor based on the synergy between optical flow and orthogonal variant moments. The bioinspired sensor has been designed for Very Large Scale Integration (VLSI) using properties of the mammalian cortical motion pathway. This sensor combines low-level primitives (optical flow and image moments) in order to produce a mid-level vision abstraction layer. The results are described trough experiments showing the validity of the proposed system and an analysis of the computational resources and performance of the applied algorithms.

A method to learn the inverse kinematics of multi-link robots by evolving neuro-controllers

A general method to learn the inverse kinematic of multi-link robots by means of neuro-controllers is presented. We can find analytical solutions for the most used and well-known robots in the literature. However, these solutions are specific to a particular robot configuration and are not generally applicable to other robot morphologies. The proposed method is general in the sense that it is independent of the robot morphology. The method is based on the evolutionary computation paradigm and works obtaining incrementally better neuro-controllers. Furthermore, the proposed method solves some specific issues in robotic neuro-controller learning: it avoids any neural network learning algorithm which relies on the classical supervised input-target learning scheme and hence it lets to obtain neuro-controllers without providing targets. It can converge beyond local optimal solutions, which is one of the main drawbacks of some neural network training algorithms based on gradient descent when applied to highly redundant robot morphologies. Furthermore, using learning algorithms such as the neuro-evolution of augmenting topologies it is also possible to learn the neural network topology which is a common source of empirical testing in neuro-controllers design. Finally, experimental results are provided when applying the method to two multi-link robot learning tasks and a comparison between structural and parametric evolutionary strategies on neuro-controllers is shown.

Dyna-H: A heuristic planning reinforcement learning algorithm applied to role-playing game strategy decision systems

In a Role-Playing Game, finding optimal trajectories is one of the most important tasks. In fact, the strategy decision system becomes a key component of a game engine. Determining the way in which decisions are taken (online, batch or simulated) and the consumed resources in decision making (e.g. execution time, memory) will influence, in mayor degree, the game performance. When classical search algorithms such as A* can be used, they are the very first option. Nevertheless, such methods rely on precise and complete models of the search space, and there are many interesting scenarios where their application is not possible. Then, model free methods for sequential decision making under uncertainty are the best choice. In this paper, we propose a heuristic planning strategy to incorporate the ability of heuristic-search in path-finding into a Dyna agent. The proposed Dyna-H algorithm, as A* does, selects branches more likely to produce outcomes than other branches. Besides, it has the advantages of being a model-free online reinforcement learning algorithm. The proposal was evaluated against the one-step Q-Learning and Dyna-Q algorithms obtaining excellent experimental results: Dyna-H significantly overcomes both methods in all experiments. We suggest also, a functional analogy between the proposed sampling from worst trajectories heuristic and the role of dreams (e.g. nightmares) in human behavior.

The kNN-TD Reinforcement Learning Algorithm

A reinforcement learning algorithm called k NN-TD is introduced. This algorithm has been developed using the classical formulation of temporal difference methods and a k -nearest neighbors scheme as its expectations memory. By means of this kind of memory the algorithm is able to generalize properly over continuous state spaces and also take benefits from collective action selection and learning processes. Furthermore, with the addition of probability traces, we obtain the k NN-TD(*** ) algorithm which exhibits a state of the art performance. Finally the proposed algorithm has been tested on a series of well known reinforcement learning problems and also at the Second Annual RL Competition with excellent results.A reinforcement learning algorithm called kNN-TD is introduced. This algorithm has been developed using the classical formulation of temporal difference methods and a k-nearest neighbors scheme as its expectations memory. By means of this kind of memory the algorithm is able to generalize properly over continuous state spaces and also take benefits from collective action selection and learning processes. Furthermore, with the addition of probability traces, we obtain the kNN-TD(λ) algorithm which exhibits a state of the art performance. Finally the proposed algorithm has been tested on a series of well known reinforcement learning problems and also at the Second Annual RL Competition with excellent results.

Ex〈α〉: An effective algorithm for continuous actions Reinforcement Learning problems

In this paper the Ex(α) Reinforcement Learning algorithm is presented. This algorithm is designed to deal with problems where the use of continuous actions have clear advantages over the use of fine grained discrete actions. This new algorithm is derived from a baseline discrete actions algorithm implemented within a kind of fc-nearest neighbors approach in order to produce a probabilistic representation of the input signal to construct robust state descriptions based on a collection (knn) of receptive field units and a probability distribution vector p(knn) over the knn collection. The baseline continuous-space-discrete-actions fcNN-TD(A) algorithm introduces probability traces as the natural adaptation of eligibility traces in the probabilistic context. Later the Ex(α)(λ) algorithm is described as an extension of the baseline algorithms. Finally experimental results are presented for two (not easy) problems such as the Cart-Pole and Helicopter Hovering.

Learning Autonomous Helicopter Flight with Evolutionary Reinforcement Learning

In this paper we present a method to obtain a near optimal neuro-controller for the autonomous helicopter flight by means of an ad hoc evolutionary reinforcement learning method. The method presented here was developed for the Second Annual Reinforcement Learning Competition (RL2008) held in Helsinki-Finland. The present work uses a Helicopter Hovering simulator created in the Stanford University that simulates a Radio Control XCell Tempest helicopter in the flight regime close to hover. The objective of the controller is to hover the helicopter by manipulating four continuous control actions based on a 12-dimensional state space.

Solving Hard Computational Problems Efficiently: Asymptotic Parametric Complexity 3-Coloring Algorithm

Many practical problems in almost all scientific and technological disciplines have been classified as computationally hard (NP-hard or even NP-complete). In life sciences, combinatorial optimization problems frequently arise in molecular biology, e.g., genome sequencing; global alignment of multiple genomes; identifying siblings or discovery of dysregulated pathways. In almost all of these problems, there is the need for proving a hypothesis about certain property of an object that can be present if and only if it adopts some particular admissible structure (an NP-certificate) or be absent (no admissible structure), however, none of the standard approaches can discard the hypothesis when no solution can be found, since none can provide a proof that there is no admissible structure. This article presents an algorithm that introduces a novel type of solution method to “efficiently” solve the graph 3-coloring problem; an NP-complete problem. The proposed method provides certificates (proofs) in both cases: present or absent, so it is possible to accept or reject the hypothesis on the basis of a rigorous proof. It provides exact solutions and is polynomial-time (i.e., efficient) however parametric. The only requirement is sufficient computational power, which is controlled by the parameter . Nevertheless, here it is proved that the probability of requiring a value of to obtain a solution for a random graph decreases exponentially: , making tractable almost all problem instances. Thorough experimental analyses were performed. The algorithm was tested on random graphs, planar graphs and 4-regular planar graphs. The obtained experimental results are in accordance with the theoretical expected results.

Helicopter flight dynamics using soft computing models

In this paper we propose a novel approach for the control of helicopters, using a flight dynamics model of the aircraft to develop reliable controllers by means of classical procedures, evolutionary either reinforcement learning techniques. Here we are presenting the method that we use to estimate the aircraft position, including the low level image processing, the hardware configuration which allows us to register the commands generated by an expert pilot using a conventional radio control (RC) transmitter, and how both variables are related by an artificial neural network (ANN).