Tiago V. Maia

From reinforcement learning models to psychiatric and neurological disorders.

Over the last decade and a half, reinforcement learning models have fostered an increasingly sophisticated understanding of the functions of dopamine and cortico-basal ganglia-thalamo-cortical (CBGTC) circuits. More recently, these models, and the insights that they afford, have started to be used to understand important aspects of several psychiatric and neurological disorders that involve disturbances of the dopaminergic system and CBGTC circuits. We review this approach and its existing and potential applications to Parkinsons disease, Tourettes syndrome, attention-deficit/hyperactivity disorder, addiction, schizophrenia and preclinical animal models used to screen new antipsychotic drugs. The approachs proven explanatory and predictive power bodes well for the continued growth of computational psychiatry and computational neurology.

The neural bases of obsessive-compulsive disorder in children and adults.

Functional imaging studies have reported with remarkable consistency hyperactivity in the orbitofrontal cortex (OFC), anterior cingulate cortex (ACC), and caudate nucleus of patients with obsessive-compulsive disorder (OCD). These findings have often been interpreted as evidence that abnormalities in cortico-basal ganglia-thalamo-cortical loops involving the OFC and ACC are causally related to OCD. This interpretation remains controversial, however, because such hyperactivity may represent either a cause or a consequence of the symptoms. This article analyzes the evidence for a causal role of these loops in producing OCD in children and adults. The article first reviews the strong evidence for anatomical abnormalities in these loops in patients with OCD. These findings are not sufficient to establish causality, however, because anatomical alterations may themselves be a consequence rather than a cause of the symptoms. The article then reviews three lines of evidence that, despite their own limitations, permit stronger causal inferences: the development of OCD following brain injury, pediatric autoimmune neuropsychiatric disorders associated with streptococcal infection, and neurosurgical lesions that attenuate OCD. Converging evidence from these various lines of research supports a causal role for the cortico-basal ganglia-thalamo-cortical loops that involve the OFC and ACC in the pathogenesis of OCD in children and adults.

Functional Disturbances Within Frontostriatal Circuits Across Multiple Childhood Psychopathologies

OBJECTIVE Neuroimaging studies of healthy individuals inform us about the normative maturation of the frontostriatal circuits that subserve self-regulatory control processes. Findings from these studies can be used as a reference frame against which to compare the aberrant development of these processes in individuals across a wide range of childhood psychopathologies. METHOD The authors reviewed extensive neuroimaging evidence for the presence of abnormalities in frontostriatal circuits in children and adults with Tourettes syndrome and obsessive-compulsive disorder (OCD) as well as a more limited number of imaging studies of adolescents and adults with anorexia nervosa or bulimia nervosa that, together, implicate dysregulation of frontostriatal control systems in the pathogenesis of these eating disorders. RESULTS The presence of an impaired capacity for self-regulatory control that derives from abnormal development of frontostriatal circuits likely interacts in similar ways with normally occurring somatic sensations and motor urges, intrusive thoughts, sensations of hunger, and preoccupation with body shape and weight to contribute, respectively, to the development of the tics of Tourettes syndrome, the obsessions of OCD, the binge eating behaviors of bulimia, and the self-starvation of anorexia. CONCLUSIONS Analogous brain mechanisms in parallel frontostriatal circuits, or even in differing portions of the same frontostriatal circuit, may underlie the differing behavioral disturbances in these multiple disorders, although further research is needed to confirm this hypothesis.

Computational psychiatry as a bridge from neuroscience to clinical applications

Translating advances in neuroscience into benefits for patients with mental illness presents enormous challenges because it involves both the most complex organ, the brain, and its interaction with a similarly complex environment. Dealing with such complexities demands powerful techniques. Computational psychiatry combines multiple levels and types of computation with multiple types of data in an effort to improve understanding, prediction and treatment of mental illness. Computational psychiatry, broadly defined, encompasses two complementary approaches: data driven and theory driven. Data-driven approaches apply machine-learning methods to high-dimensional data to improve classification of disease, predict treatment outcomes or improve treatment selection. These approaches are generally agnostic as to the underlying mechanisms. Theory-driven approaches, in contrast, use models that instantiate prior knowledge of, or explicit hypotheses about, such mechanisms, possibly at multiple levels of analysis and abstraction. We review recent advances in both approaches, with an emphasis on clinical applications, and highlight the utility of combining them.

The neural circuits that generate tics in Tourette's syndrome.

OBJECTIVE The purpose of this study was to examine neural activity and connectivity within cortico-striato-thalamo-cortical circuits and to reveal circuit-based neural mechanisms that govern tic generation in Tourettes syndrome. METHOD Functional magnetic resonance imaging data were acquired from 13 individuals with Tourettes syndrome and 21 healthy comparison subjects during spontaneous or simulated tics. Independent component analysis with hierarchical partner matching was used to isolate neural activity within functionally distinct regions of cortico-striato-thalamo-cortical circuits. Granger causality was used to investigate causal interactions among these regions. RESULTS The Tourettes syndrome group exhibited stronger neural activity and interregional causality than healthy comparison subjects throughout all portions of the motor pathway, including the sensorimotor cortex, putamen, pallidum, and substantia nigra. Activity in these areas correlated positively with the severity of tic symptoms. Activity within the Tourettes syndrome group was stronger during spontaneous tics than during voluntary tics in the somatosensory and posterior parietal cortices, putamen, and amygdala/hippocampus complex, suggesting that activity in these regions may represent features of the premonitory urges that generate spontaneous tic behaviors. In contrast, activity was weaker in the Tourettes syndrome group than in the healthy comparison group within portions of cortico-striato-thalamo-cortical circuits that exert top-down control over motor pathways (the caudate and anterior cingulate cortex), and progressively less activity in these regions accompanied more severe tic symptoms, suggesting that faulty activity in these circuits may result in their failure to control tic behaviors or the premonitory urges that generate them. CONCLUSIONS Our findings, taken together, suggest that tics are caused by the combined effects of excessive activity in motor pathways and reduced activation in control portions of cortico-striato-thalamo-cortical circuits.

Reinforcement Learning, Conditioning, and the Brain: Successes and Challenges

The field of reinforcement learning has greatly influenced the neuroscientific study of conditioning. This article provides an introduction to reinforcement learning followed by an examination of the successes and challenges using reinforcement learning to understand the neural bases of conditioning. Successes reviewed include (1) the mapping of positive and negative prediction errors to the firing of dopamine neurons and neurons in the lateral habenula, respectively; (2) the mapping of model-based and model-free reinforcement learning to associative and sensorimotor cortico-basal ganglia-thalamo-cortical circuits, respectively; and (3) the mapping of actor and critic to the dorsal and ventral striatum, respectively. Challenges reviewed consist of several behavioral and neural findings that are at odds with standard reinforcement-learning models, including, among others, evidence for hyperbolic discounting and adaptive coding. The article suggests ways of reconciling reinforcement-learning models with many of the challenging findings, and highlights the need for further theoretical developments where necessary. Additional information related to this study may be downloaded from http://cabn.psychonomic-journals.org/content/supplemental.

Abnormal Amygdalar Activation and Connectivity in Adolescents With Attention-Deficit/Hyperactivity Disorder

OBJECTIVE Emotional reactivity is one of the most disabling symptoms associated with attention-deficit/hyperactivity disorder (ADHD). We aimed to identify neural substrates associated with emotional reactivity and to assess the effects of stimulants on those substrates. METHOD We used functional magnetic resonance imaging (fMRI) to assess neural activity in adolescents with (n = 15) and without (n = 15) ADHD while they performed a task involving the subliminal presentation of fearful faces. Using dynamic causal modeling, we also examined the effective connectivity of two regions associated with emotional reactivity, i.e., the amygdala and the lateral prefrontal cortex (LPFC). The participants with ADHD underwent scanning both on and off stimulant medication in a counterbalanced fashion. RESULTS During the task, we found that activity in the right amygdala was greater in adolescents with ADHD than in control subjects. In addition, in adolescents with ADHD, greater connectivity was detected between the amygdala and LPFC. Stimulants had a normalizing effect on both the activity in the right amygdala and the connectivity between the amygdala and LPFC. CONCLUSIONS Our findings demonstrate that in adolescents with ADHD, a neural substrate of fear processing is atypical, as is the connectivity between the amygdala and LPFC. These findings suggest possible neural substrates for the emotional reactivity that is often present in youths with ADHD, and provide putative neural targets for the development of novel therapeutic interventions for this condition.

The somatic marker hypothesis: still many questions but no answers: Response to Bechara et al

The writing of this article was supported by a fellowship from the Calouste Gulbenkian Foundation (Portugal) to T.V.M. and by NIMH grant MH64445 to J.L.M.

Reduced functional connectivity within the limbic cortico-striato-thalamo-cortical loop in unmedicated adults with obsessive-compulsive disorder

Cortico‐striato‐thalamo‐cortical (CSTC) loops project from the cortex to the striatum, then from the striatum to the thalamus via the globus pallidus, and finally from the thalamus back to the cortex again. These loops have been implicated in Obsessive‐Compulsive Disorder (OCD) with particular focus on the limbic CSTC loop, which encompasses the orbitofrontal and anterior cingulate cortices, as well as the ventral striatum. Resting state functional‐connectivity MRI (rs‐fcMRI) studies, which examine temporal correlations in neural activity across brain regions at rest, have examined CSTC loop connectivity in patients with OCD and suggest hyperconnectivity within these loops in medicated adults with OCD. We used rs‐fcMRI to examine functional connectivity within CSTC loops in unmedicated adults with OCD (n = 23) versus healthy controls (HCs) (n = 20). Contrary to prior rs‐fcMRI studies in OCD patients on medications that report hyperconnectivity in the limbic CSTC loop, we found that compared with HCs, unmedicated OCD participants had reduced connectivity within the limbic CSTC loop. Exploratory analyses revealed that reduced connectivity within the limbic CSTC loop correlated with OCD symptom severity in the OCD group. Our finding of limbic loop hypoconnectivity in unmedicted OCD patients highlights the potential confounding effects of antidepressants on connectivity measures and the value of future examinations of the effects of pharmacological and/or behavioral treatments on limbic CSTC loop connectivity. Hum Brain Mapp 35:2852–2860, 2014.

Two-factor theory, the actor-critic model, and conditioned avoidance

Two-factor theory (Mowrer, 1947, 1951, 1956) remains one of the most influential theories of avoidance, but it is at odds with empirical findings that demonstrate sustained avoidance responding in situations in which the theory predicts that the response should extinguish. This article shows that the well-known actor-critic model seamlessly addresses the problems with two-factor theory, while simultaneously being consistent with the core ideas that underlie that theory. More specifically, the article shows that (1) the actor-critic model bears striking similarities to two-factor theory and explains all of the empirical phenomena that two-factor theory explains, in much the same way, and (2) there are subtle but important differences between the actor-critic model and two-factor theory, which result in the actor-critic model predicting the persistence of avoidance responses that is found empirically.