Ivan Herreros

Consensus Paper: Towards a Systems-Level View of Cerebellar Function: the Interplay Between Cerebellum, Basal Ganglia, and Cortex

Despite increasing evidence suggesting the cerebellum works in concert with the cortex and basal ganglia, the nature of the reciprocal interactions between these three brain regions remains unclear. This consensus paper gathers diverse recent views on a variety of important roles played by the cerebellum within the cerebello-basal ganglia-thalamo-cortical system across a range of motor and cognitive functions. The paper includes theoretical and empirical contributions, which cover the following topics: recent evidence supporting the dynamical interplay between cerebellum, basal ganglia, and cortical areas in humans and other animals; theoretical neuroscience perspectives and empirical evidence on the reciprocal influences between cerebellum, basal ganglia, and cortex in learning and control processes; and data suggesting possible roles of the cerebellum in basal ganglia movement disorders. Although starting from different backgrounds and dealing with different topics, all the contributors agree that viewing the cerebellum, basal ganglia, and cortex as an integrated system enables us to understand the function of these areas in radically different ways. In addition, there is unanimous consensus between the authors that future experimental and computational work is needed to understand the function of cerebellar-basal ganglia circuitry in both motor and non-motor functions. The paper reports the most advanced perspectives on the role of the cerebellum within the cerebello-basal ganglia-thalamo-cortical system and illustrates other elements of consensus as well as disagreements and open questions in the field.

2013 Special Issue: Nucleo-olivary inhibition balances the interaction between the reactive and adaptive layers in motor control

In the acquisition of adaptive motor reflexes to aversive stimuli, the cerebellar output fulfills a double purpose: it controls a motor response and it relays a sensory prediction. However, the question of how these two apparently incompatible goals might be achieved by the same cerebellar area remains open. Here we propose a solution where the inhibition of the Inferior Olive (IO) by the cerebellar Deep Nuclei (DN) translates the motor command signal into a sensory prediction allowing a single cerebellar area to simultaneously tackle both aspects of the problem: execution and prediction. We demonstrate that having a graded error signal, the gain of the Nucleo-Olivary Inhibition (NOI) balances the generation of the response between the cerebellar and the reflexive controllers or, in other words, between the adaptive and the reactive layers of behavior. Moreover, we show that the resulting system is fully autonomous and can either acquire or erase adaptive responses according to their utility.

A VLSI Field-Programmable Mixed-Signal Array to Perform Neural Signal Processing and Neural Modeling in a Prosthetic System

A very-large-scale integration field-programmable mixed-signal array specialized for neural signal processing and neural modeling has been designed. This has been fabricated as a core on a chip prototype intended for use in an implantable closed-loop prosthetic system aimed at rehabilitation of the learning of a discrete motor response. The chosen experimental context is cerebellar classical conditioning of the eye-blink response. The programmable system is based on the intimate mixing of switched capacitor analog techniques with low speed digital computation; power saving innovations within this framework are presented. The utility of the system is demonstrated by the implementation of a motor classical conditioning model applied to eye-blink conditioning in real time with associated neural signal processing. Paired conditioned and unconditioned stimuli were repeatedly presented to an anesthetized rat and recordings were taken simultaneously from two precerebellar nuclei. These paired stimuli were detected in real time from this multichannel data. This resulted in the acquisition of a trigger for a well-timed conditioned eye-blink response, and repetition of unpaired trials constructed from the same data led to the extinction of the conditioned response trigger, compatible with natural cerebellar learning in awake animals.

Speed generalization capabilities of a cerebellar model on a rapid navigation task

The cerebellum is a brain structure necessary for skilled motor behaviour and has a well understood and repetitive architecture. Such an architecture inspired the Marr-Albus-Ito theory of cerebellar learning, that provides an explanation for the acquisition of motor skills by the cerebellum. Numerous computational models inspired in such a theory have already been employed in robotic tasks. Here we look into one of the suggested roles of the cerebellum, the replacement of reflexes by anticipatory actions and we apply it to a robot navigation task. The acquisition of anticipatory actions has been thoroughly studied in the field of classical conditioning. Of particular interest is the so-called CS-intensity effect, an effect that links the rapidity of execution of an anticipatory protective action, the Conditioned Response (CR), to the intensity of a predictive signal, the Conditioning Stimulus (CS). We propose that the CS-intensity effect implements a built-in sensory-motor contingency that allows to carry over a skill learned in a safe and easy context, e.g., turning at slow velocity, to a more difficult one, e.g., a turning at a faster speed. We demonstrate this hypothesis in a series of experiments where a robot has to navigate a track that has a turn. We show that after being trained at a slow velocity, by means of the CS-intensity effect, the cerebellar controller modulates the turning such that its onset anticipates as the robot speed increases. Ultimately, through incremental learning, this generalization allows the robot to learn to navigate the track at its maximum speed.

Behavioral rehabilitation of the eye closure reflex in senescent rats using a real-time biosignal acquisition system

In this paper the replacement of a lost learning function of rats through a computer-based real-time recording and feedback system is shown. In an experiment two recording electrodes and one stimulation electrode were implanted in an anesthetized rat. During a classical-conditioning paradigm, which includes tone and airpuff stimulation, biosignals were recorded and the stimulation events detected. A computational model of the cerebellum acquired the association between the stimuli and gave feedback to the brain of the rat using deep brain stimulation in order to close the eyelid of the rat. The study shows that replacement of a lost brain function using a direct bidirectional interface to the brain is realizable and can inspire future research for brain rehabilitation.

On Three Categories of Conscious Machines

Reviewing recent closely related developments at the crossroads of biomedical engineering, artificial intelligence and biomimetic technology, in this paper, we attempt to distinguish phenomenological consciousness into three categories based on embodiment: one that is embodied by biological agents, another by artificial agents and a third that results from collective phenomena in complex dynamical systems. Though this distinction by itself is not new, such a classification is useful for understanding differences in design principles and technology necessary to engineer conscious machines. It also allows one to zero-in on minimal features of phenomenological consciousness in one domain and map on to their counterparts in another. For instance, awareness and metabolic arousal are used as clinical measures to assess levels of consciousness in patients in coma or in a vegetative state. We discuss analogous abstractions of these measures relevant to artificial systems and their manifestations. This is particularly relevant in the light of recent developments in deep learning and artificial life.

Optimization of the Anticipatory Reflexes of a Computational Model of the Cerebellum

The cerebellum is involved in avoidance learning tasks, where anticipatory actions are developed to protect against aversive stimuli. In the execution and acquisition of discrete actions we can distinguish errors of omission and commission due to a failure to execute a required defensive Conditioned Response (CR) to avoid an aversive Unconditioned Stimulus (US), and the energy expenditure of triggering an unnecessary CR in the absence of a US respectively. Hence, a motor learning cost function must consider both these components of performance and energy expenditure. Unlike remaining noxious stimuli, unnecessary actions are not directly sensed by the cerebellum. It has been suggested that the Nucleo-Olivary Inhibition (NOI) serves to internally rely information about these needless protective actions. Here we argue that the function of the NOI can be interpreted in broader terms as a signal that is used to learn optimal actions in terms of cost. We work with a computational model of the cerebellum to address: (i) how can the optimum balance between remaining aversive stimuli and preventing effort be found, and (ii) how can the cerebellum use the overall cost information to establish this optimum balance through the adjusting of the gain of the NOI. In this paper we derive the value of the NOI that minimizes the overall cost and propose a learning rule for the cerebellum through which this value is reached. We test this rule in a collision avoidance task performed by a simulated robot.

A Cerebellar Neuroprosthetic System: Computational Architecture and in vivo Test

Emulating the input–output functions performed by a brain structure opens the possibility for developing neuroprosthetic systems that replace damaged neuronal circuits. Here, we demonstrate the feasibility of this approach by replacing the cerebellar circuit responsible for the acquisition and extinction of motor memories. Specifically, we show that a rat can undergo acquisition, retention, and extinction of the eye-blink reflex even though the biological circuit responsible for this task has been chemically inactivated via anesthesia. This is achieved by first developing a computational model of the cerebellar microcircuit involved in the acquisition of conditioned reflexes and training it with synthetic data generated based on physiological recordings. Secondly, the cerebellar model is interfaced with the brain of an anesthetized rat, connecting the model’s inputs and outputs to afferent and efferent cerebellar structures. As a result, we show that the anesthetized rat, equipped with our neuroprosthetic system, can be classically conditioned to the acquisition of an eye-blink response. However, non-stationarities in the recorded biological signals limit the performance of the cerebellar model. Thus, we introduce an updated cerebellar model and validate it with physiological recordings showing that learning becomes stable and reliable. The resulting system represents an important step toward replacing lost functions of the central nervous system via neuroprosthetics, obtained by integrating a synthetic circuit with the afferent and efferent pathways of a damaged brain region. These results also embody an early example of science-based medicine, where on the one hand the neuroprosthetic system directly validates a theory of cerebellar learning that informed the design of the system, and on the other one it takes a step toward the development of neuro-prostheses that could recover lost learning functions in animals and, in the longer term, humans.

A forward model at Purkinje cell synapses facilitates cerebellar anticipatory control

How does our motor system solve the problem of anticipatory control in spite of a wide spectrum of response dynamics from different musculo-skeletal systems, transport delays as well as response latencies throughout the central nervous system? To a great extent, our highly-skilled motor responses are a result of a reactive feedback system, originating in the brain-stem and spinal cord, combined with a feed-forward anticipatory system, that is adaptively fine-tuned by sensory experience and originates in the cerebellum. Based on that interaction we design the counterfactual predictive control (CFPC) architecture, an anticipatory adaptive motor control scheme, in which a feed-forward module, based on the cerebellum, steers an error feedback controller with counterfactual error signals. Those are signals that trigger reactions as actual errors would, but that do not code for any current of forthcoming errors. In order to determine the optimal learning strategy, we derive a novel learning rule for the feed-forward module that involves an eligibility trace and operates at the synaptic level. In particular, our eligibility trace provides a mechanism beyond co-incidence detection in that it convolves a history of prior synaptic inputs with error signals. In the context of cerebellar physiology, this solution implies that Purkinje cell synapses should generate eligibility traces using a forward model of the system being controlled. From an engineering perspective, CFPC provides a general-purpose anticipatory control architecture equipped with a learning rule that exploits the full dynamics of the closed-loop system.

The Role of a Cerebellum-Driven Perceptual Prediction within a Robotic Postural Task

Postural adjustments are acquired compensatory and anticipatory motor responses maintaining balance and equilibrium against self-induced or external perturbations. It has been proposed that the cerebellum could be involved in issuing such predictive motor actions. However, it remains unclear what strategy is adopted by the brain in order to make such prediction and how anticipatory and compensatory components are integrated into a single response. Within this study we are interested in the computational mechanisms underlying the acquisition of anticipatory responses in a postural task. We compare two alternative architectures representing two different hypotheses: anticipation either as sensory-to-motor association or as sensory-to-sensory association. We propose to use a cerebellar model to control the acquisition of an adaptive motor response in a simulated robotic setup. We devise a scenario where a cart-pole robot is trained to predict a perturbation and issue an anticipatory action to minimize the disturbance on its state of equilibrium. Our results show that a cerebellum based architecture can efficiently learn to reduce errors through anticipation. We also suggest that a sensory-to-sensory prediction could be less expensive in terms of energy cost and more robust when events violate the acquired prediction.