Michael Moutoussis

The anatomy of choice: active inference and agency

This paper considers agency in the setting of embodied or active inference. In brief, we associate a sense of agency with prior beliefs about action and ask what sorts of beliefs underlie optimal behavior. In particular, we consider prior beliefs that action minimizes the Kullback–Leibler (KL) divergence between desired states and attainable states in the future. This allows one to formulate bounded rationality as approximate Bayesian inference that optimizes a free energy bound on model evidence. We show that constructs like expected utility, exploration bonuses, softmax choice rules and optimism bias emerge as natural consequences of this formulation. Previous accounts of active inference have focused on predictive coding and Bayesian filtering schemes for minimizing free energy. Here, we consider variational Bayes as an alternative scheme that provides formal constraints on the computational anatomy of inference and action—constraints that are remarkably consistent with neuroanatomy. Furthermore, this scheme contextualizes optimal decision theory and economic (utilitarian) formulations as pure inference problems. For example, expected utility theory emerges as a special case of free energy minimization, where the sensitivity or inverse temperature (of softmax functions and quantal response equilibria) has a unique and Bayes-optimal solution—that minimizes free energy. This sensitivity corresponds to the precision of beliefs about behavior, such that attainable goals are afforded a higher precision or confidence. In turn, this means that optimal behavior entails a representation of confidence about outcomes that are under an agents control.

The anatomy of choice: dopamine and decision-making.

This paper considers goal-directed decision-making in terms of embodied or active inference. We associate bounded rationality with approximate Bayesian inference that optimizes a free energy bound on model evidence. Several constructs such as expected utility, exploration or novelty bonuses, softmax choice rules and optimism bias emerge as natural consequences of free energy minimization. Previous accounts of active inference have focused on predictive coding. In this paper, we consider variational Bayes as a scheme that the brain might use for approximate Bayesian inference. This scheme provides formal constraints on the computational anatomy of inference and action, which appear to be remarkably consistent with neuroanatomy. Active inference contextualizes optimal decision theory within embodied inference, where goals become prior beliefs. For example, expected utility theory emerges as a special case of free energy minimization, where the sensitivity or inverse temperature (associated with softmax functions and quantal response equilibria) has a unique and Bayes-optimal solution. Crucially, this sensitivity corresponds to the precision of beliefs about behaviour. The changes in precision during variational updates are remarkably reminiscent of empirical dopaminergic responses—and they may provide a new perspective on the role of dopamine in assimilating reward prediction errors to optimize decision-making.

Adolescence is associated with genomically patterned consolidation of the hubs of the human brain connectome

Significance Adolescence is a period of human brain growth and high incidence of mental health disorders. Here, we show consistently in two MRI cohorts that human brain changes in adolescence were concentrated on the more densely connected hubs of the connectome (i.e., association cortical regions that mediated efficient connectivity throughout the human brain structural network). Hubs were less myelinated at 14 y but had faster rates of myelination and cortical shrinkage in the 14- to 24-y period. This topologically focused process of cortical consolidation was associated with expression of genes enriched for normal synaptic and myelin-related processes and risk of schizophrenia. Consolidation of anatomical network hubs could be important for normal and clinically disordered adolescent brain development. How does human brain structure mature during adolescence? We used MRI to measure cortical thickness and intracortical myelination in 297 population volunteers aged 14–24 y old. We found and replicated that association cortical areas were thicker and less myelinated than primary cortical areas at 14 y. However, association cortex had faster rates of shrinkage and myelination over the course of adolescence. Age-related increases in cortical myelination were maximized approximately at the internal layer of projection neurons. Adolescent cortical myelination and shrinkage were coupled and specifically associated with a dorsoventrally patterned gene expression profile enriched for synaptic, oligodendroglial- and schizophrenia-related genes. Topologically efficient and biologically expensive hubs of the brain anatomical network had greater rates of shrinkage/myelination and were associated with overexpression of the same transcriptional profile as cortical consolidation. We conclude that normative human brain maturation involves a genetically patterned process of consolidating anatomical network hubs. We argue that developmental variation of this consolidation process may be relevant both to normal cognitive and behavioral changes and the high incidence of schizophrenia during human brain adolescence.

Persecutory delusions and the conditioned avoidance paradigm: Towards an integration of the psychology and biology of paranoia

Introduction. Theories of delusions often underplay the role of their content. With respect to persecutory delusions, taking threat as fundamental suggests that models of threat-related, aversive learning, such as the Conditioned Avoidance Response (CAR) task, might offer valid insights into the underlying normal and abnormal processes. In this study, we reappraise the psychological significance of the CAR model of antipsychotic drug action; and we relate this to contemporary psychological theories of paranoia. Methods. Review and synthesis of literature. Results. Anticipation and recall of aversive events are abnormally accentuated in paranoia. Safety (avoidance) behaviours may help perpetuate and fix persecutory ideas by preventing their disconfirmation. In addition, patients may explain negative events in a paranoid way instead of making negative self-attributions (i.e., in an attempt to maintain self-esteem). This defensive function only predominates in the overtly psychotic patients. The “safety behaviours” of paranoid patients, their avoidance of negative self-attributions, and the antiparanoid effect of antipsychotic medication all resonate with aspects of the CAR. Conclusions. The CAR appears to activate some normal psychological and biological processes that are pathologically activated in paranoid psychosis. Paranoid psychological defences may be a result of basic aversive learning mechanisms, which are accentuated during acute psychosis.

A temporal difference account of avoidance learning

Aversive processing plays a central role in human phobic fears and may also be important in some symptoms of psychosis. We developed a temporal-difference model of the conditioned avoidance response, an important experimental model for aversive learning which is also a central pharmacological model of psychosis. In the model, dopamine neurons reported outcomes that were better than the learner expected, typically coming from reaching safety states, and thus controlled the acquisition of a suitable policy. The model accounts for normal conditioned avoidance learning, the persistence of responding in extinction, and critical effects of dopamine blockade, notably that subjects experiencing shocks under dopamine blockade, and hence failing to avoid them, nevertheless develop avoidance responses when both shocks and dopamine blockade are subsequently removed. These postulated roles of dopamine in aversive learning can thus account for many of the effects of dopaminergic modulation seen in laboratory models of psychopathological processes.

Learning-Induced Plasticity in Medial Prefrontal Cortex Predicts Preference Malleability

Summary Learning induces plasticity in neuronal networks. As neuronal populations contribute to multiple representations, we reasoned plasticity in one representation might influence others. We used human fMRI repetition suppression to show that plasticity induced by learning another individual’s values impacts upon a value representation for oneself in medial prefrontal cortex (mPFC), a plasticity also evident behaviorally in a preference shift. We show this plasticity is driven by a striatal “prediction error,” signaling the discrepancy between the other’s choice and a subject’s own preferences. Thus, our data highlight that mPFC encodes agent-independent representations of subjective value, such that prediction errors simultaneously update multiple agents’ value representations. As the resulting change in representational similarity predicts interindividual differences in the malleability of subjective preferences, our findings shed mechanistic light on complex human processes such as the powerful influence of social interaction on beliefs and preferences.

Bayesian inferences about the self (and others): a review.

Highlights • People may use Bayesian inference to update their own self-representation.• Self- and other-representations may help predict outcomes of social interactions.• The value of an outcome is essentially the prior belief that it can be achieved.• ‘Active inference’ uses free-energy-minimization to achieve desirable outcomes.• A positive self-representation may be a desirable outcome of active inference.

Reflection impulsivity in binge drinking: behavioural and volumetric correlates.

The degree to which an individual accumulates evidence prior to making a decision, also known as reflection impulsivity, can be affected in psychiatric disorders. Here, we study decisional impulsivity in binge drinkers, a group at elevated risk for developing alcohol use disorders, comparing two tasks assessing reflection impulsivity and a delay discounting task, hypothesizing impairments in both subtypes of impulsivity. We also assess volumetric correlates of reflection impulsivity focusing on regions previously implicated in functional magnetic resonance imaging studies. Sixty binge drinkers and healthy volunteers were tested using two different information‐gathering paradigms: the beads task and the Information Sampling Task (IST). The beads task was analysed using a behavioural approach and a Bayesian model of decision making. Delay discounting was assessed using the Monetary Choice Questionnaire. Regression analyses of primary outcomes were conducted with voxel‐based morphometry analyses. Binge drinkers sought less evidence prior to decision in the beads task compared with healthy volunteers in both the behavioural and computational modelling analysis. There were no group differences in the IST or delay discounting task. Greater impulsivity as indexed by lower evidence accumulation in the beads task was associated with smaller dorsolateral prefrontal cortex and inferior parietal volumes. In contrast, greater impulsivity as indexed by lower evidence accumulation in the IST was associated with greater dorsal cingulate and precuneus volumes. Binge drinking is characterized by impaired reflection impulsivity suggesting a deficit in deciding on the basis of future outcomes that are more difficult to represent. These findings emphasize the role of possible therapeutic interventions targeting decision‐making deficits.

Computational psychiatry of ADHD: Neural gain impairments across marrian levels of analysis

Attention-deficit hyperactivity disorder (ADHD), one of the most common psychiatric disorders, is characterised by unstable response patterns across multiple cognitive domains. However, the neural mechanisms that explain these characteristic features remain unclear. Using a computational multilevel approach, we propose that ADHD is caused by impaired gain modulation in systems that generate this phenotypic increased behavioural variability. Using Marrs three levels of analysis as a heuristic framework, we focus on this variable behaviour, detail how it can be explained algorithmically, and how it might be implemented at a neural level through catecholamine influences on corticostriatal loops. This computational, multilevel, approach to ADHD provides a framework for bridging gaps between descriptions of neuronal activity and behaviour, and provides testable predictions about impaired mechanisms.

Active inference, evidence accumulation, and the urn task

Deciding how much evidence to accumulate before making a decision is a problem we and other animals often face, but one that is not completely understood. This issue is particularly important because a tendency to sample less information (often known as reflection impulsivity) is a feature in several psychopathologies, such as psychosis. A formal understanding of information sampling may therefore clarify the computational anatomy of psychopathology. In this theoretical letter, we consider evidence accumulation in terms of active (Bayesian) inference using a generic model of Markov decision processes. Here, agents are equipped with beliefs about their own behavior—in this case, that they will make informed decisions. Normative decision making is then modeled using variational Bayes to minimize surprise about choice outcomes. Under this scheme, different facets of belief updating map naturally onto the functional anatomy of the brain (at least at a heuristic level). Of particular interest is the key role played by the expected precision of beliefs about control, which we have previously suggested may be encoded by dopaminergic neurons in the midbrain. We show that manipulating expected precision strongly affects how much information an agent characteristically samples, and thus provides a possible link between impulsivity and dopaminergic dysfunction. Our study therefore represents a step toward understanding evidence accumulation in terms of neurobiologically plausible Bayesian inference and may cast light on why this process is disordered in psychopathology.