Martí Sánchez-Fibla

ALLOSTATIC CONTROL FOR ROBOT BEHAVIOR REGULATION: A COMPARATIVE RODENT-ROBOT STUDY

Rodents are optimal real-world foragers that regulate internal states maintaining a dynamic stability with their surroundings. How these internal drive based behaviors are regulated remains unclear. Based on the physiological notion of allostasis, we investigate a minimal control system able to approximate their behavior. Allostasis is the process of achieving stability with the environment through change, opposed to homeostasis which achieves it through constancy. Following this principle, the so-called allostatic control system orchestrates the interaction of the homeostatic modules by changing their desired values in order to achieve stability. We use a minimal number of subsystems and estimate the model parameters from rat behavioral data in three experimental setups: free exploration, presence of reward, delivery of cues with reward predictive value. From this analysis, we show that a rat is influenced by the shape of the arena in terms of its openness. We then use the estimated model configurations to control a simulated and real robot which captures essential properties of the observed rat behavior. The allostatic reactive control model is proposed as an augmentation of the Distributed Adaptive Control architecture and provides a further contribution towards the realization of an artificial rodent.

The acquisition of intentionally indexed and object centered affordance gradients: A biomimetic controller and mobile robotics benchmark

We introduce affordance gradients (AGs), continuous sensorimotor structures that allow to predict the consequences of the agents actions on the state of the environment. AGs allow to generalize among never performed actions and compress all possible consequences of the action state space. AGs also provide a way of estimating the world state after several interactions of the agent with objects. We validate the notion of AGs using benchmarks designed for mobile robotics that we solve using E-puck robot simulations: learn the affordances of several objects, push an object along a predefined trajectory and place an object in at a target position and orientation. We are interested in the neurophysiological basis of affordances and how they can be inserted in a sensorimotor loop with memory structures like the one proposed by the DAC architecture. We show that AGs provide a generalization of the perception-action couplets stored in memory and learned by they adaptive layer of DAC.

Speed generalization capabilities of a cerebellar model on a rapid navigation task

The cerebellum is a brain structure necessary for skilled motor behaviour and has a well understood and repetitive architecture. Such an architecture inspired the Marr-Albus-Ito theory of cerebellar learning, that provides an explanation for the acquisition of motor skills by the cerebellum. Numerous computational models inspired in such a theory have already been employed in robotic tasks. Here we look into one of the suggested roles of the cerebellum, the replacement of reflexes by anticipatory actions and we apply it to a robot navigation task. The acquisition of anticipatory actions has been thoroughly studied in the field of classical conditioning. Of particular interest is the so-called CS-intensity effect, an effect that links the rapidity of execution of an anticipatory protective action, the Conditioned Response (CR), to the intensity of a predictive signal, the Conditioning Stimulus (CS). We propose that the CS-intensity effect implements a built-in sensory-motor contingency that allows to carry over a skill learned in a safe and easy context, e.g., turning at slow velocity, to a more difficult one, e.g., a turning at a faster speed. We demonstrate this hypothesis in a series of experiments where a robot has to navigate a track that has a turn. We show that after being trained at a slow velocity, by means of the CS-intensity effect, the cerebellar controller modulates the turning such that its onset anticipates as the robot speed increases. Ultimately, through incremental learning, this generalization allows the robot to learn to navigate the track at its maximum speed.

Distributed Adaptive Control: A Proposal on the Neuronal Organization of Adaptive Goal Oriented Behavior

In behavioral motor coordination and interaction it is a fundamental challenge how an agent can learn to perceive and act in unknown and dynamic environments. At present, it is not clear how an agent can – without any explicitly predefined knowledge – acquire internal representations of the world while interacting with the environment. To meet this challenge, we propose a biologically based cognitive architecture called Distributed Adaptive Control (DAC). DAC is organized in three different, tightly coupled, layers of control: reactive, adaptive and contextual. DAC based systems are self-contained and fully grounded, meaning that they autonomously generate representations of their primary sensory inputs, hence bootstrapping their behavior form simple to advance interactions. Following this approach, we have previously identified a novel environmentally mediated feedback loop in the organization of perception and behavior, i.e. behavioral feedback. Additionally, we could demonstrated that the dynamics of the memory structure of DAC, acquired during a foraging task, are equivalent to a Bayesian description of foraging. In this chapter we present DAC in a concise form and show how it is allowing us to extend the different subsystems to more biophysical detailed models. These further developments of the DAC architecture, not only allow to better understand the biological systems, but moreover advance DACs behavioral capabilities and generality.

The Hierarchical Accumulation of Knowledge in the Distributed Adaptive Control Architecture

Animals acquire knowledge as they interact with the world. Several authors define this acquisition as a chain of transformations: data is acquired and converted into information that is converted into knowledge. Moreover, theories on cumulative learning suggest that different cognitive layers accumulate this knowledge, building highly complex skills from low complexity ones. The biologically based Distributed Adaptive Control cognitive architecture (DAC) has been proposed as a cumulative learning system. DAC contains different layers of control: reactive, adaptive and contextual. This hierarchical organization allows for acquisition of knowledge in a bottom-up interaction, i.e. sampled data is transformed into knowledge. DAC has already been used as a framework to investigate fundamental problems encountered in biology. Here we describe the DAC architecture and present some studies focused on its highest cognitive layer where knowledge is constructed and used. We investigate the roles of reactive and contextual control depending on the characteristics and complexity of the tasks. We also show how multi-sensor information could be integrated in order to acquire and use knowledge optimally. Finally, we discuss the possible problems of cumulative learning and the adopted solutions in the context of DAC.

The Role of a Cerebellum-Driven Perceptual Prediction within a Robotic Postural Task

Postural adjustments are acquired compensatory and anticipatory motor responses maintaining balance and equilibrium against self-induced or external perturbations. It has been proposed that the cerebellum could be involved in issuing such predictive motor actions. However, it remains unclear what strategy is adopted by the brain in order to make such prediction and how anticipatory and compensatory components are integrated into a single response. Within this study we are interested in the computational mechanisms underlying the acquisition of anticipatory responses in a postural task. We compare two alternative architectures representing two different hypotheses: anticipation either as sensory-to-motor association or as sensory-to-sensory association. We propose to use a cerebellar model to control the acquisition of an adaptive motor response in a simulated robotic setup. We devise a scenario where a cart-pole robot is trained to predict a perturbation and issue an anticipatory action to minimize the disturbance on its state of equilibrium. Our results show that a cerebellum based architecture can efficiently learn to reduce errors through anticipation. We also suggest that a sensory-to-sensory prediction could be less expensive in terms of energy cost and more robust when events violate the acquired prediction.

The neuronal substrate underlying order and interval representations in sequential tasks: A biologically based robot study

Sequence learning tasks depend on the ability to acquire and control the order of actions and their proper timing. Several studies have shown that in sequence learning different areas of the brain are involved when recalling the order of actions and their proper interval. One hypothesis proposes that two separate areas of the brain interact with each other, one computes order while the other would compute the interval. A second hypothesis proposes that one area computes both, order and interval. To better understand how this computation of order and interval might be realized by the brain, we developed a robot based architecture and investigated the behavioral and architectural implications of these two hypothesis: one or two neuronal areas computing order and interval. Using a sequence learning foraging task we show that performance is enhanced in case of distributed processes. However, we show that as a drawback, explicit interval information can not be reconstructed.

Autonomous development of turn-taking behaviors in agent populations: A computational study

We provide a computational model showing how turn-taking behaviors can self-organize out of sensorimotor interactions between vocalizing agents. Recent hypotheses propose that turn-taking behaviors in certain primate species emerge from a need to maintain vocal contact in a group (e.g. in dense environments preventing visual contact). In this context, vocalizations can convey information about the presence of each group member and taking turns allow to minimize the vocal signal interferences. We consider agents equipped with a cognitive architecture based on two coupled control loops: a reactive one implementing a basic regulatory behavior to maintain vocal listening and an adaptive one learning an action policy to maximize vocal contact among group members. We show that the reactive process bootstraps the adaptive learning to converge toward a collective turn-taking strategy. This model provides a computational support to the hypothesis that turn-taking can emerge from functional constraints related to group cohesion and inter-individual vocal signal interferences. We suggest future directions of research to understand how social behaviors can result from sensorimotor interactions.

The complementary roles of allostatic and contextual control systems in foraging tasks

To survive in an unknown environment an animal has to learn how to reach specific goal states. The animal is firstly guided by its reactive behavior motivated by its internal needs. After exploring the environment, contextual information can be used to optimally fulfill these internal needs. However, how a reactive and a contextual control system complement each other is still a fundamental question. Here, we address this problem from the perspective of the Distributed Adaptive Control architecture (DAC). We extend DACs reactive layer with an allostatic control system and integrate it with its contextual control layer. Through robot foraging tasks we test the properties of the allostatic and contextual cozntrol systems and their interaction. We assess how they scale with task complexity. In particular, we show that the behavior generated by the contextual control layer is of particular importance when the system is facing conflict situations.

Acquisition of Synergistic Motor Responses through Cerebellar Learning in a Robotic Postural Task

Coordination of synergistic movements is a crucial aspect of goal oriented motor behavior in postural control. It has been proposed that the cerebellum could be involved in the acquisition of adaptive fine-tuned motor responses. However, it remains unclear whether motor patterns and action sequences can be learned as a result of recurrent connections among multiple cerebellar microcircuits. Within this study we hypothesize that such link could be found in the Nucleo-Pontine projection and we investigate the behavioral advantages of cerebellar driven synergistic motor responses in a robotic postural task. We devise a scenario where a double-joint cart-pole robot has to learn to stand and balance interconnected segments by issuing multiple actions in order to minimize the deviation from a state of equilibrium. Our results show that a cerebellum based architecture can efficiently learn to reduce errors through well-timed motor coordination. We also suggest that such strategy could reduce energy cost by progressively synchronizing multiple joints movements.