Jordi-Ysard Puigbò

DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge About the World and the Self

This paper introduces a cognitive architecture for a humanoid robot to engage in a proactive, mixed-initiative exploration and manipulation of its environment, where the initiative can originate from both human and robot. The framework, based on a biologically grounded theory of the brain and mind, integrates a reactive interaction engine, a number of state-of-the-art perceptual and motor learning algorithms, as well as planning abilities and an autobiographical memory. The architecture as a whole drives the robot behavior to solve the symbol grounding problem, acquire language capabilities, execute goal-oriented behavior, and express a verbal narrative of its own experience in the world. We validate our approach in human-robot interaction experiments with the iCub humanoid robot, showing that the proposed cognitive architecture can be applied in real time within a realistic scenario and that it can be used with naive users.

Towards Self-controlled Robots Through Distributed Adaptive Control

Robots, as well as machine learning algorithms, have proven to be, unlike human beings, very sensitive to errors and failure. Artificial intelligence and machine learning are nowadays the main source of algorithms that drive cognitive robotics research. The advances in the fields have been huge during the last year, beating expert-human performance in video games, an achievement that was unthinkable a few years ago. Still, performance has been assessed by external measures not necessarily fit to the problem to solve, what lead to shameful failure on some specific tasks. We propose that the way to achieve human-like robustness in performance is to consider the self of the agent as the real source of self-evaluated error. This offers a solution to acting when information or resources are scarce and learning speed is important. This paper details our extension of the cognitive architecture DAC to control embodied agents and robots, through self-generated signals, from needs, drives, self-generated value and goals.

Synaptogenesis: Constraining Synaptic Plasticity Based on a Distance Rule

Neural models, artificial or biologically grounded, have been used for understanding the nature of learning mechanisms as well as for applied tasks. The study of such learning systems has been typically centered on the identification or extraction of the most relevant features that will help to solve a task. Recently, convolutional networks, deep architectures and huge reservoirs have shown impressive results in tasks ranging from speech recognition to visual classification or emotion perception. With the accumulated momentum of such large-scale architectures, the importance of imposing sparsity on the networks to differentiate contexts has been rising. We present a biologically grounded system that imposes physical and local constraints to these architectures in the form of synaptogenesis, or synapse generation. This method guarantees sparsity and promotes the acquisition of experience-relevant, topologically-organized and more diverse features.

Towards a Two-Phase Model of Sensor and Motor Learning

The cerebellum has an important role on motor learning. How sensory data arrives to the cerebellum is hardly understood. A two-phase model is proposed to understand how raw sensory data is processed to facilitate cerebellar predictive learning. Different candidates are presented for guiding the perceptual learning phase grounded on the role of the amygdala. A hebbian learning based computational model is presented with some preliminary results.

Are Brains Computers, Emulators or Simulators?

There has been intense debate on the question of whether the brain is a computer. If so, that challenge is to show that all cognitive processes can be described by algorithms running on a universal Turing machine. By extension that implies consciousness is a computational process. Both Penrose and Searle have vehemently argued against this view, proposing that consciousness is a fundamentally non-computational process [10]. Even proponents of the brain as a computer metaphor such a Dennett agree that the organizational architecture of the brain is unlike any computing system ever conceived, possibly alluding to non-classical computational processes [6]. The latter class of processes veer away from any program that can be encoded by Church’s lambda calculus. In fact, such a program would have to be based on non-classical logic (either semi-classical or quantum). But quantum logic or machines that might implement them typically are not meant for solving the same type of problems that a classical computer solves (nor are they necessarily faster for any given problem). We will argue that machines implementing non-classical logic might be better suited for simulation rather than computation (a la Turing). It is thus reasonable to pit simulation as an alternative to computation and ask whether the brain, rather than computing, is simulating a model of the world in order to make predictions and guide behavior. If so, this suggests a hardware supporting dynamics more akin to a quantum many-body field theory.

Learning Modular Sequences in the Striatum

The execution of habitual actions is thought to rely on the exploitation of procedural motor memories. These memories encode motor commands as organized in functional sequences with well defined boundaries in the Striatum. Here, we present a biophysical model of the striatal network composed by inhibitory medium spiny neurons (MSNs) governed by anti-hebbian STDP. We show that these two features allow for learning an arbitrary sequence through multiple exposures to cortical inputs and reproducing it under a single, non-specific excitatory drive. Our results shed light on the computational properties of biologically plausible inhibitory networks and suggest a simple, yet effective mechanism of behavioral control through striatal circuits.

Challenges of Machine Learning for Living Machines

Machine Learning algorithms (and in particular Reinforcement Learning (RL)) have proved very successful in recent years. These have managed to achieve super-human performance in many different tasks, from video-games to board-games and complex cognitive tasks such as path-planning or Theory of Mind (ToM) on artificial agents. Nonetheless, this super-human performance is also super-artificial. Despite some metrics are better than what a human can achieve (i.e. cumulative reward), in less common metrics (i.e. time to learning asymptote) the performance is significantly worse. Moreover, the means by which those are achieved fail to extend our understanding of the human or mammal brain. Moreover, most approaches used are based on black-box optimization, making any comparison beyond performance (e.g. at the architectural level) difficult. In this position paper, we review the origins of reinforcement learning and propose its extension with models of learning derived from fear and avoidance behaviors. We argue that avoidance-based mechanisms are required when training on embodied, situated systems to ensure fast and safe convergence and potentially overcome some of the current limitations of the RL paradigm.

Modeling the Opponent’s Action Using Control-Based Reinforcement Learning

In this paper, we propose an alternative to model-free reinforcement learning approaches that recently have demonstrated Theory-of-Mind like behaviors. We propose a game theoretic approach to the problem in which pure RL has demonstrated to perform below the standards of human-human interaction. In this context, we propose alternative learning architectures that complement basic RL models with the ability to predict the other’s actions. This architecture is tested in different scenarios where agents equipped with similar or varying capabilities compete in a social game. Our different interaction scenarios suggest that our model-based approaches are especially effective when competing against models of equivalent complexity, in contrast to our previous results with more basic predictive architectures. We conclude that the evolution of mechanisms that allow for the control of other agents provide different kinds of advantages that can become significant when interacting with different kinds of agents. We argue that no single proposed addition to the learning architecture is sufficient to optimize performance in these scenarios, but a combination of the different mechanisms suggested is required to achieve near-optimal performance in any case.

iCub-HRI: A Software Framework for Complex Human–Robot Interaction Scenarios on the iCub Humanoid Robot

Generating complex, human-like behavior in a humanoid robot like the iCub requires the integration of a wide range of open source components and a scalable cognitive architecture. Hence, we present the iCub-HRI library which provides convenience wrappers for components related to perception (object recognition, agent tracking, speech recognition, and touch detection), object manipulation (basic and complex motor actions), and social interaction (speech synthesis and joint attention) exposed as a C++ library with bindings for Java (allowing to use iCub-HRI within Matlab) and Python. In addition to previously integrated components, the library allows for simple extension to new components and rapid prototyping by adapting to changes in interfaces between components. We also provide a set of modules which make use of the library, such as a high-level knowledge acquisition module and an action recognition module. The proposed architecture has been successfully employed for a complex human–robot interaction scenario involving the acquisition of language capabilities, execution of goal-oriented behavior and expression of a verbal narrative of the robot’s experience in the world. Accompanying this paper is a tutorial which allows a subset of this interaction to be reproduced. The architecture is aimed at researchers familiarizing themselves with the iCub ecosystem, as well as expert users, and we expect the library to be widely used in the iCub community.

Cerebellar-inspired learning rule for gain adaptation of feedback controllers

The cerebellum is a crucial brain structure in enabling precise motor control in animals. Recent advances suggest that the timing of the plasticity rule of Purkinje cells, the main cells of the cerebellum, is matched to behavioral function. Simultaneously, counter-factual predictive control (CFPC), a cerebellar-based control scheme, has shown that the optimal rule for learning feed-forward action in an adaptive filter playing the role of the cerebellum must include a forward model of the system controlled. Here we show how the same learning rule obtained in CFPC, which we term as Model-enhanced least mean squares (ME-LMS), emerges in the problem of learning the gains of a feedback controller. To that end, we frame a model-reference adaptive control (MRAC) problem and derive an adaptive control scheme treating the gains of a feedback controller as if they were the weights of an adaptive linear unit. Our results demonstrate that the approach of controlling plasticity with a forward model of the subsystem controlled can provide a solution to a wide set of adaptive control problems.