Benoît Girard

The Psikharpax project: Towards building an artificial rat

Abstract Drawing inspiration from biology, the Psikharpax project aims at endowing a robot with a sensory-motor equipment and a neural control architecture that will afford some of the capacities of autonomy and adaptation that are exhibited by real rats. The paper summarizes the current state of achievement of the project. It successively describes the robots future sensors and actuators, and several biomimetic models of the anatomy and physiology of structures in the rats brain, like the hippocampus and the basal ganglia, which have already been at work on various robots, and that make navigation and action selection possible. Preliminary results on the implementation of learning mechanisms in these structures are also presented. Finally, the article discusses the potential benefits that a biologically inspired approach affords to traditional autonomous robotics.

2008 Special Issue: Where neuroscience and dynamic system theory meet autonomous robotics: A contracting basal ganglia model for action selection

Action selection, the problem of choosing what to do next, is central to any autonomous agent architecture. We use here a multi-disciplinary approach at the convergence of neuroscience, dynamical system theory and autonomous robotics, in order to propose an efficient action selection mechanism based on a new model of the basal ganglia. We first describe new developments of contraction theory regarding locally projected dynamical systems. We exploit these results to design a stable computational model of the cortico-baso-thalamo-cortical loops. Based on recent anatomical data, we include usually neglected neural projections, which participate in performing accurate selection. Finally, the efficiency of this model as an autonomous robot action selection mechanism is assessed in a standard survival task. The model exhibits valuable dithering avoidance and energy-saving properties, when compared with a simple if-then-else decision rule.

Actor-Critic Models of Reinforcement Learning in the Basal Ganglia: From Natural to Artificial Rats

Since 1995, numerous Actor–Critic architectures for reinforcement learning have been proposed as models of dopamine-like reinforcement learning mechanisms in the rat’s basal ganglia. However, these models were usually tested in different tasks, and it is then difficult to compare their efficiency for an autonomous animat. We present here the comparison of four architectures in an animat as it per forms the same reward-seeking task. This will illustrate the consequences of different hypotheses about the management of different Actor sub-modules and Critic units, and their more or less autono mously determined coordination. We show that the classical method of coordination of modules by mixture of experts, depending on each module’s performance, did not allow solving our task. Then we address the question of which principle should be applied efficiently to combine these units. Improve ments for Critic modeling and accuracy of Actor–Critic models for a natural task are finally discussed in the perspective of our Psikharpax project—an artificial rat having to survive autonomously in unpre dictable environments.

A biologically inspired meta-control navigation system for the Psikharpax rat robot

A biologically inspired navigation system for the mobile rat-like robot named Psikharpax is presented, allowing for self-localization and autonomous navigation in an initially unknown environment. The ability of parts of the model (e.g. the strategy selection mechanism) to reproduce rat behavioral data in various maze tasks has been validated before in simulations. But the capacity of the model to work on a real robot platform had not been tested. This paper presents our work on the implementation on the Psikharpax robot of two independent navigation strategies (a place-based planning strategy and a cue-guided taxon strategy) and a strategy selection meta-controller. We show how our robot can memorize which was the optimal strategy in each situation, by means of a reinforcement learning algorithm. Moreover, a context detector enables the controller to quickly adapt to changes in the environment-recognized as new contexts-and to restore previously acquired strategy preferences when a previously experienced context is recognized. This produces adaptivity closer to rat behavioral performance and constitutes a computational proposition of the role of the rat prefrontal cortex in strategy shifting. Moreover, such a brain-inspired meta-controller may provide an advancement for learning architectures in robotics.

Path planning versus cue responding: a bio-inspired model of switching between navigation strategies

In this article, we describe a new computational model of switching between path-planning and cue-guided navigation strategies. It is based on three main assumptions: (i) the strategies are mediated by separate memory systems that learn independently and in parallel; (ii) the learning algorithms are different in the two memory systems—the cue-guided strategy uses a temporal-difference (TD) learning rule to approach a visible goal, whereas the path-planning strategy relies on a place-cell-based graph-search algorithm to learn the location of a hidden goal; (iii) a strategy selection mechanism uses TD-learning rule to choose the most successful strategy based on past experience. We propose a novel criterion for strategy selection based on the directions of goal-oriented movements suggested by the different strategies. We show that the selection criterion based on this “common currency” is capable of choosing the best among TD-learning and planning strategies and can be used to solve navigational tasks in continuous state and action spaces. The model has been successfully applied to reproduce rat behavior in two water-maze tasks in which the two strategies were shown to interact. The model was used to analyze competitive and cooperative interactions between different strategies during these tasks as well as relative influence of different types of sensory cues.

Geometry of the superior colliculus mapping and efficient oculomotor computation

Numerous brain regions encode variables using spatial distribution of activity in neuronal maps. Their specific geometry is usually explained by sensory considerations only. We provide here, for the first time, a theory involving the motor function of the superior colliculus to explain the geometry of its maps. We use six hypotheses in accordance with neurobiology to show that linear and logarithmic mappings are the only ones compatible with the generation of saccadic motor command. This mathematical proof gives a global coherence to the neurobiological studies on which it is based. Moreover, a new solution to the problem of saccades involving both colliculi is proposed. Comparative simulations show that it is more precise than the classical one.

Integration of Navigation and Action Selection Functionalities in a Computational Model of Cortico-Basal-Ganglia-Thalamo-Cortical Loops

This article describes a biomimetic control architecture affording an animat both action selection and navigation functionalities. It satisfies the survival constraint of an artificial metabolism and supports several complementary navigation strategies. It builds upon an action selection model based on the basal ganglia of the vertebrate brain, using two interconnected cortico-basal-ganglia–thalamo-cortical loops: A ventral one concerned with appetitive actions and a dorsal one dedicated to consummatory actions. The performances of the resulting model are evaluated in simulation. The experiments assess the prolonged survival permitted by the use of high-level navigation strategies and the com plementarity of navigation strategies in dynamic environments. The correctness of the behavioral choices in situations of antagonistic or synergetic internal states are also tested. Finally, the modeling choices are discussed with regard to their biomimetic plausibility, while the experimental results are estimated in terms of animat adaptivity.

Sustainable computational science: the ReScience initiative

Computer science offers a large set of tools for prototyping, writing, running, testing, validating, sharing and reproducing results, however computational science lags behind. In the best case, authors may provide their source code as a compressed archive and they may feel confident their research is reproducible. But this is not exactly true. James Buckheit and David Donoho proposed more than two decades ago that an article about computational results is advertising, not scholarship. The actual scholarship is the full software environment, code, and data that produced the result. This implies new workflows, in particular in peer-reviews. Existing journals have been slow to adapt: source codes are rarely requested, hardly ever actually executed to check that they produce the results advertised in the article. ReScience is a peer-reviewed journal that targets computational research and encourages the explicit replication of already published research, promoting new and open-source implementations in order to ensure that the original research can be replicated from its description. To achieve this goal, the whole publishing chain is radically different from other traditional scientific journals. ReScience resides on GitHub where each new implementation of a computational study is made available together with comments, explanations, and software tests.

Implementation of a neurophysiological model of saccadic eye movements on an anthropomorphic robotic head

In this paper we investigated the relevance of a robotic implementation in the development and validation of a neurophysiological model of the generation of saccadic eye movements. To this aim, a well-characterized model of the brainstem saccadic circuitry was implemented on a humanoid robot head with 7 degrees of freedom (DOFs), which was designed to mimic the human head in terms of the physical dimensions (i.e. geometry and masses), the kinematics (i.e. number of DOFs and ranges of motion), the dynamics (i.e. velocities and accelerations), and the functionality (i.e. the ocular movements of vergence, smooth pursuit and saccades). Our implementation makes the robot head execute saccadic eye movements upon a visual stimulus appearing in the periphery of the robot visual field, by reproducing the following steps: projection or the camera images onto collicular images, according to the modeled mapping between the retina and the superior colliculus (SC); transformation of the retinotopic coordinates of the stimulus obtained in the camera reference frame into their corresponding projections on the SC; spatio-temporal transformation of these coordinates according to what is known to happen in the brainstem saccade burst generator of primates; and execution of the eye movement by controlling one eye motor of the robot, in velocity. The capabilities of the robot head to execute saccadic movements have been tested with respect to the neurophysiological model implemented, in view of the use of this robotic implementation for validating and tuning the model itself, in further focused experimental trials

A biologically constrained model of the whole basal ganglia addressing the paradoxes of connections and selection

The basal ganglia nuclei form a complex network of nuclei often assumed to perform selection, yet their individual roles and how they influence each other is still largely unclear. In particular, the ties between the external and internal parts of the globus pallidus are paradoxical, as anatomical data suggest a potent inhibitory projection between them while electrophysiological recordings indicate that they have similar activities. Here we introduce a theoretical study that reconciles both views on the intra-pallidal projection, by providing a plausible characterization of the relationship between the external and internal globus pallidus. Specifically, we developed a mean-field model of the whole basal ganglia, whose parameterization is optimized to respect best a collection of numerous anatomical and electrophysiological data. We first obtained models respecting all our constraints, hence anatomical and electrophysiological data on the intrapallidal projection are globally consistent. This model furthermore predicts that both aforementioned views about the intra-pallidal projection may be reconciled when this projection is weakly inhibitory, thus making it possible to support similar neural activity in both nuclei and for the entire basal ganglia to select between actions. Second, we predicts that afferent projections are substantially unbalanced towards the external segment, as it receives the strongest excitation from STN and the weakest inhibition from the striatum. Finally, our study strongly suggests that the intrapallidal connection pattern is not focused but diffuse, as this latter pattern is more efficient for the overall selection performed in the basal ganglia.