Vincenzo G. Fiore

Keep focussing: striatal dopamine multiple functions resolved in a single mechanism tested in a simulated humanoid robot.

The effects of striatal dopamine (DA) on behavior have been widely investigated over the past decades, with “phasic” burst firings considered as the key expression of a reward prediction error responsible for reinforcement learning. Less well studied is “tonic” DA, where putative functions include the idea that it is a regulator of vigor, incentive salience, disposition to exert an effort and a modulator of approach strategies. We present a model combining tonic and phasic DA to show how different outflows triggered by either intrinsically or extrinsically motivating stimuli dynamically affect the basal ganglia by impacting on a selection process this system performs on its cortical input. The model, which has been tested on the simulated humanoid robot iCub interacting with a mechatronic board, shows the putative functions ascribed to DA emerging from the combination of a standard computational mechanism coupled to a differential sensitivity to the presence of DA across the striatum.

Computational psychiatry of ADHD: Neural gain impairments across marrian levels of analysis

Attention-deficit hyperactivity disorder (ADHD), one of the most common psychiatric disorders, is characterised by unstable response patterns across multiple cognitive domains. However, the neural mechanisms that explain these characteristic features remain unclear. Using a computational multilevel approach, we propose that ADHD is caused by impaired gain modulation in systems that generate this phenotypic increased behavioural variability. Using Marrs three levels of analysis as a heuristic framework, we focus on this variable behaviour, detail how it can be explained algorithmically, and how it might be implemented at a neural level through catecholamine influences on corticostriatal loops. This computational, multilevel, approach to ADHD provides a framework for bridging gaps between descriptions of neuronal activity and behaviour, and provides testable predictions about impaired mechanisms.

Evolutionarily conserved mechanisms for the selection and maintenance of behavioural activity

Survival and reproduction entail the selection of adaptive behavioural repertoires. This selection manifests as phylogenetically acquired activities that depend on evolved nervous system circuitries. Lorenz and Tinbergen already postulated that heritable behaviours and their reliable performance are specified by genetically determined programs. Here we compare the functional anatomy of the insect central complex and vertebrate basal ganglia to illustrate their role in mediating selection and maintenance of adaptive behaviours. Comparative analyses reveal that central complex and basal ganglia circuitries share comparable lineage relationships within clusters of functionally integrated neurons. These clusters are specified by genetic mechanisms that link birth time and order to their neuronal identities and functions. Their subsequent connections and associated functions are characterized by similar mechanisms that implement dimensionality reduction and transition through attractor states, whereby spatially organized parallel-projecting loops integrate and convey sensorimotor representations that select and maintain behavioural activity. In both taxa, these neural systems are modulated by dopamine signalling that also mediates memory-like processes. The multiplicity of similarities between central complex and basal ganglia suggests evolutionarily conserved computational mechanisms for action selection. We speculate that these may have originated from ancestral ground pattern circuitries present in the brain of the last common ancestor of insects and vertebrates.

Corticolimbic catecholamines in stress: a computational model of the appraisal of controllability

Appraisal of a stressful situation and the possibility to control or avoid it is thought to involve frontal-cortical mechanisms. The precise mechanism underlying this appraisal and its translation into effective stress coping (the regulation of physiological and behavioural responses) are poorly understood. Here, we propose a computational model which involves tuning motivational arousal to the appraised stressing condition. The model provides a causal explanation of the shift from active to passive coping strategies, i.e. from a condition characterised by high motivational arousal, required to deal with a situation appraised as stressful, to a condition characterised by emotional and motivational withdrawal, required when the stressful situation is appraised as uncontrollable/unavoidable. The model is motivated by results acquired via microdialysis recordings in rats and highlights the presence of two competing circuits dominated by different areas of the ventromedial prefrontal cortex: these are shown having opposite effects on several subcortical areas, affecting dopamine outflow in the striatum, and therefore controlling motivation. We start by reviewing published data supporting structure and functioning of the neural model and present the computational model itself with its essential neural mechanisms. Finally, we show the results of a new experiment, involving the condition of repeated inescapable stress, which validate most of the model’s predictions.

The “Mechatronic Board”: A Tool to Study Intrinsic Motivations in Humans, Monkeys, and Humanoid Robots

In this chapter the design and fabrication of a new mechatronic platform (called “mechatronic board”) for behavioural analysis of children, non-human primates, and robots are presented and discussed. The platform is the result of a multidisciplinary design approach which merges indications coming from neuroscientists, psychologists, primatologists, roboticists, and bioengineers, with the main goal of studying learning mechanisms driven by intrinsic motivations and curiosity. This chapter firstly introduces the main requirements of the platform, coming from the different needs of the experiments involving the different types of participants. Then, it provides a detailed analysis of the main features of the mechatronic board, focusing on its key aspects which allow the study of intrinsically motivated learning in children and non-human primates. Finally, it shows some preliminary results on curiosity-driven learning coming from pilot experiments involving children, capuchin monkeys, and a computational model of the behaviour of these organisms tested with a humanoid robot (the iCub robot). These experiments investigate the capacity of children, capuchin monkeys, and a computational model implemented on the iCub robot to learn action-outcome contingencies on the basis of intrinsic motivations.

Changing pattern in the basal ganglia: motor switching under reduced dopaminergic drive

Action selection in the basal ganglia is often described within the framework of a standard model, associating low dopaminergic drive with motor suppression. Whilst powerful, this model does not explain several clinical and experimental data, including varying therapeutic efficacy across movement disorders. We tested the predictions of this model in patients with Parkinson’s disease, on and off subthalamic deep brain stimulation (DBS), focussing on adaptive sensory-motor responses to a changing environment and maintenance of an action until it is no longer suitable. Surprisingly, we observed prolonged perseverance under on-stimulation, and high inter-individual variability in terms of the motor selections performed when comparing the two conditions. To account for these data, we revised the standard model exploring its space of parameters and associated motor functions and found that, depending on effective connectivity between external and internal parts of the globus pallidus and saliency of the sensory input, a low dopaminergic drive can result in increased, dysfunctional, motor switching, besides motor suppression. This new framework provides insight into the biophysical mechanisms underlying DBS, allowing a description in terms of alteration of the signal-to-baseline ratio in the indirect pathway, which better account of known electrophysiological data in comparison with the standard model.

A lineage-related reciprocal inhibition circuitry for sensory-motor action selection

The insect central complex and vertebrate basal ganglia are forebrain centres involved in selection and maintenance of behavioural actions. However, little is known about the formation of the underlying circuits, or how they integrate sensory information for motor actions. Here, we show that paired embryonic neuroblasts generate central complex ring neurons that mediate sensory-motor transformation and action selection in Drosophila. Lineage analysis resolves four ring neuron subtypes, R1-R4, that form GABAergic inhibition circuitry among inhibitory sister cells. Genetic manipulations, together with functional imaging, demonstrate subtype-specific R neurons mediate the selection and maintenance of behavioural activity. A computational model substantiates genetic and behavioural observations suggesting that R neuron circuitry functions as salience detector using competitive inhibition to amplify, maintain or switch between activity states. The resultant gating mechanism translates facilitation, inhibition and disinhibition of behavioural activity as R neuron functions into selection of motor actions and their organisation into action sequences.

In silico interrogation of insect central complex suggests computational roles for the ellipsoid body in spatial navigation

The central complex in the insect brain is a composite of midline neuropils involved in processing sensory cues and mediating behavioral outputs to orchestrate spatial navigation. Despite recent advances, however, the neural mechanisms underlying sensory integration and motor action selections have remained largely elusive. In particular, it is not yet understood how the central complex exploits sensory inputs to realize motor functions associated with spatial navigation. Here we report an in silico interrogation of central complex-mediated spatial navigation with a special emphasis on the ellipsoid body. Based on known connectivity and function, we developed a computational model to test how the local connectome of the central complex can mediate sensorimotor integration to guide different forms of behavioral outputs. Our simulations show integration of multiple sensory sources can be effectively performed in the ellipsoid body. This processed information is used to trigger continuous sequences of action selections resulting in self-motion, obstacle avoidance and the navigation of simulated environments of varying complexity. The motor responses to perceived sensory stimuli can be stored in the neural structure of the central complex to simulate navigation relying on a collective of guidance cues, akin to sensory-driven innate or habitual behaviors. By comparing behaviors under different conditions of accessible sources of input information, we show the simulated insect computes visual inputs and body posture to estimate its position in space. Finally, we tested whether the local connectome of the central complex might also allow the flexibility required to recall an intentional behavioral sequence, among different courses of actions. Our simulations suggest that the central complex can encode combined representations of motor and spatial information to pursue a goal and thus successfully guide orientation behavior. Together, the observed computational features identify central complex circuitry, and especially the ellipsoid body, as a key neural correlate involved in spatial navigation.

Value encoding in the globus pallidus: FMRI reveals an interaction effect between reward and dopamine drive

&NA; The external part of the globus pallidus (GPe) is a core nucleus of the basal ganglia (BG) whose activity is disrupted under conditions of low dopamine release, as in Parkinsons disease. Current models assume decreased dopamine release in the dorsal striatum results in deactivation of dorsal GPe, which in turn affects motor expression via a regulatory effect on other nuclei of the BG. However, recent studies in healthy and pathological animal models have reported neural dynamics that do not match with this view of the GPe as a relay in the BG circuit. Thus, the computational role of the GPe in the BG is still to be determined. We previously proposed a neural model that revisits the functions of the nuclei of the BG, and this model predicts that GPe encodes values which are amplified under a condition of low striatal dopaminergic drive. To test this prediction, we used an fMRI paradigm involving a within‐subject placebo‐controlled design, using the dopamine antagonist risperidone, wherein healthy volunteers performed a motor selection and maintenance task under low and high reward conditions. ROI‐based fMRI analysis revealed an interaction between reward and dopamine drive manipulations, with increased BOLD activity in GPe in a high compared to low reward condition, and under risperidone compared to placebo. These results confirm the core prediction of our computational model, and provide a new perspective on neural dynamics in the BG and their effects on motor selection and cognitive disorders. HighlightsWe investigate the representation of action‐state values in the basal ganglia.Value representation is enhanced in the GPe under reduced dopaminergic drive.Value representation is enhanced in the SNr under basal dopaminergic drive.The results validate a proposed neural model of the basal ganglia.

Addiction in a Bounded Rational Model: Role of Environment and its Exploration

Several decision-making vulnerabilities have been identified as underlying causes for addictive behaviours, or the repeated execution of stereotyped actions despite their adverse consequences. These vulnerabilities are mostly associated with brain alterations caused by the consumption of substances of abuse. However, addiction can also happen in the absence of a pharmacological component, such as seen in pathological gambling and videogaming. We use a new reinforcement learning model to highlight a previously neglected vulnerability that we suggest interacts with those already identified, whilst playing a prominent role in non-pharmacological forms of addiction. Specifically, we show that a duallearning system (i.e. combining model-based and model-free) can be vulnerable to highly rewarding, but suboptimal actions, that are followed by a complex ramification of stochastic adverse effects. This phenomenon is caused by the overload of the capabilities of an agent, as time and cognitive resources required for exploration, deliberation, situation recognition, and habit formation, all increase as a function of the depth and richness of detail of an environment. Furthermore, the cognitive overload can be aggravated due to alterations (e.g. caused by stress) in the bounded rationality, i.e. the limited amount of resources available for the model-based component, in turn increasing the agent’s chances to develop or maintain addictive behaviours. Our study demonstrates that, independent of drug consumption, addictive behaviours can arise in the interaction between the environmental complexity and the biologically finite resources available to explore and represent it.