Hanneke E. M. den Ouden

Nonlinear dynamic causal models for fMRI

Models of effective connectivity characterize the influence that neuronal populations exert over each other. Additionally, some approaches, for example Dynamic Causal Modelling (DCM) and variants of Structural Equation Modelling, describe how effective connectivity is modulated by experimental manipulations. Mathematically, both are based on bilinear equations, where the bilinear term models the effect of experimental manipulations on neuronal interactions. The bilinear framework, however, precludes an important aspect of neuronal interactions that has been established with invasive electrophysiological recording studies; i.e., how the connection between two neuronal units is enabled or gated by activity in other units. These gating processes are critical for controlling the gain of neuronal populations and are mediated through interactions between synaptic inputs (e.g. by means of voltage-sensitive ion channels). They represent a key mechanism for various neurobiological processes, including top-down (e.g. attentional) modulation, learning and neuromodulation. This paper presents a nonlinear extension of DCM that models such processes (to second order) at the neuronal population level. In this way, the modulation of network interactions can be assigned to an explicit neuronal population. We present simulations and empirical results that demonstrate the validity and usefulness of this model. Analyses of synthetic data showed that nonlinear and bilinear mechanisms can be distinguished by our extended DCM. When applying the model to empirical fMRI data from a blocked attention to motion paradigm, we found that attention-induced increases in V5 responses could be best explained as a gating of the V1-->V5 connection by activity in posterior parietal cortex. Furthermore, we analysed fMRI data from an event-related binocular rivalry paradigm and found that interactions amongst percept-selective visual areas were modulated by activity in the middle frontal gyrus. In both practical examples, Bayesian model selection favoured the nonlinear models over corresponding bilinear ones.

A Dual Role for Prediction Error in Associative Learning

Confronted with a rich sensory environment, the brain must learn statistical regularities across sensory domains to construct causal models of the world. Here, we used functional magnetic resonance imaging and dynamic causal modeling (DCM) to furnish neurophysiological evidence that statistical associations are learnt, even when task-irrelevant. Subjects performed an audio-visual target-detection task while being exposed to distractor stimuli. Unknown to them, auditory distractors predicted the presence or absence of subsequent visual distractors. We modeled incidental learning of these associations using a Rescorla–Wagner (RW) model. Activity in primary visual cortex and putamen reflected learning-dependent surprise: these areas responded progressively more to unpredicted, and progressively less to predicted visual stimuli. Critically, this prediction-error response was observed even when the absence of a visual stimulus was surprising. We investigated the underlying mechanism by embedding the RW model into a DCM to show that auditory to visual connectivity changed significantly over time as a function of prediction error. Thus, consistent with predictive coding models of perception, associative learning is mediated by prediction-error dependent changes in connectivity. These results posit a dual role for prediction-error in encoding surprise and driving associative plasticity.

Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning

In Bayesian brain theories, hierarchically related prediction errors (PEs) play a central role for predicting sensory inputs and inferring their underlying causes, e.g., the probabilistic structure of the environment and its volatility. Notably, PEs at different hierarchical levels may be encoded by different neuromodulatory transmitters. Here, we tested this possibility in computational fMRI studies of audio-visual learning. Using a hierarchical Bayesian model, we found that low-level PEs about visual stimulus outcome were reflected by widespread activity in visual and supramodal areas but also in the midbrain. In contrast, high-level PEs about stimulus probabilities were encoded by the basal forebrain. These findings were replicated in two groups of healthy volunteers. While our fMRI measures do not reveal the exact neuron types activated in midbrain and basal forebrain, they suggest a dichotomy between neuromodulatory systems, linking dopamine to low-level PEs about stimulus outcome and acetylcholine to more abstract PEs about stimulus probabilities.

How Prediction Errors Shape Perception, Attention, and Motivation

Prediction errors (PE) are a central notion in theoretical models of reinforcement learning, perceptual inference, decision-making and cognition, and prediction error signals have been reported across a wide range of brain regions and experimental paradigms. Here, we will make an attempt to see the forest for the trees and consider the commonalities and differences of reported PE signals in light of recent suggestions that the computation of PE forms a fundamental mode of brain function. We discuss where different types of PE are encoded, how they are generated, and the different functional roles they fulfill. We suggest that while encoding of PE is a common computation across brain regions, the content and function of these error signals can be very different and are determined by the afferent and efferent connections within the neural circuitry in which they arise.

Observing the observer (I): meta-bayesian models of learning and decision-making.

In this paper, we present a generic approach that can be used to infer how subjects make optimal decisions under uncertainty. This approach induces a distinction between a subjects perceptual model, which underlies the representation of a hidden “state of affairs” and a response model, which predicts the ensuing behavioural (or neurophysiological) responses to those inputs. We start with the premise that subjects continuously update a probabilistic representation of the causes of their sensory inputs to optimise their behaviour. In addition, subjects have preferences or goals that guide decisions about actions given the above uncertain representation of these hidden causes or state of affairs. From a Bayesian decision theoretic perspective, uncertain representations are so-called “posterior” beliefs, which are influenced by subjective “prior” beliefs. Preferences and goals are encoded through a “loss” (or “utility”) function, which measures the cost incurred by making any admissible decision for any given (hidden) state of affair. By assuming that subjects make optimal decisions on the basis of updated (posterior) beliefs and utility (loss) functions, one can evaluate the likelihood of observed behaviour. Critically, this enables one to “observe the observer”, i.e. identify (context- or subject-dependent) prior beliefs and utility-functions using psychophysical or neurophysiological measures. In this paper, we describe the main theoretical components of this meta-Bayesian approach (i.e. a Bayesian treatment of Bayesian decision theoretic predictions). In a companion paper (‘Observing the observer (II): deciding when to decide’), we describe a concrete implementation of it and demonstrate its utility by applying it to simulated and real reaction time data from an associative learning task.

Aversive pavlovian control of instrumental behavior in humans

Adaptive behavior involves interactions between systems regulating Pavlovian and instrumental control of actions. Here, we present the first investigation of the neural mechanisms underlying aversive Pavlovian–instrumental transfer using fMRI in humans. Recent evidence indicates that these Pavlovian influences on instrumental actions are action-specific: Instrumental approach is invigorated by appetitive Pavlovian cues but inhibited by aversive Pavlovian cues. Conversely, instrumental withdrawal is inhibited by appetitive Pavlovian cues but invigorated by aversive Pavlovian cues. We show that BOLD responses in the amygdala and the nucleus accumbens were associated with behavioral inhibition by aversive Pavlovian cues, irrespective of action context. Furthermore, BOLD responses in the ventromedial prefrontal cortex differed between approach and withdrawal actions. Aversive Pavlovian conditioned stimuli modulated connectivity between the ventromedial prefrontal cortex and the caudate nucleus. These results show that action-specific aversive control of instrumental behavior involves the modulation of fronto-striatal interactions by Pavlovian conditioned stimuli.

Adaptive and aberrant reward prediction signals in the human brain

Theories of the positive symptoms of schizophrenia hypothesize a role for aberrant reinforcement signaling driven by dysregulated dopamine transmission. Recently, we provided evidence of aberrant reward learning in symptomatic, but not asymptomatic patients with schizophrenia, using a novel paradigm, the Salience Attribution Test (SAT). The SAT is a probabilistic reward learning game that employs cues that vary across task-relevant and task-irrelevant dimensions; it provides behavioral indices of adaptive and aberrant reward learning. As an initial step prior to future clinical studies, here we used functional magnetic resonance imaging to examine the neural basis of adaptive and aberrant reward learning during the SAT in healthy volunteers. As expected, cues associated with high relative to low reward probabilities elicited robust hemodynamic responses in a network of structures previously implicated in motivational salience; the midbrain, in the vicinity of the ventral tegmental area, and regions targeted by its dopaminergic projections, i.e. medial dorsal thalamus, ventral striatum and prefrontal cortex (PFC). Responses in the medial dorsal thalamus and polar PFC were strongly correlated with the degree of adaptive reward learning across participants. Finally, and most importantly, differential dorsolateral PFC and middle temporal gyrus (MTG) responses to cues with identical reward probabilities were very strongly correlated with the degree of aberrant reward learning. Participants who showed greater aberrant learning exhibited greater dorsolateral PFC responses, and reduced MTG responses, to cues erroneously inferred to be less strongly associated with reward. The results are discussed in terms of their implications for different theories of associative learning.

A hemodynamic model for layered BOLD signals

High-resolution blood oxygen level dependent (BOLD) functional magnetic resonance imaging (fMRI) at the sub-millimeter scale has become feasible with recent advances in MR technology. In principle, this would enable the study of layered cortical circuits, one of the fundaments of cortical computation. However, the spatial layout of cortical blood supply may become an important confound at such high resolution. In particular, venous blood draining back to the cortical surface perpendicularly to the layered structure is expected to influence the measured responses in different layers. Here, we present an extension of a hemodynamic model commonly used for analyzing fMRI data (in dynamic causal models or biophysical network models) that accounts for such blood draining effects by coupling local hemodynamics across layers. We illustrate the properties of the model and its inversion by a series of simulations and show that it successfully captures layered fMRI data obtained during a simple visual experiment. We conclude that for future studies of the dynamics of layered neuronal circuits with high-resolution fMRI, it will be pivotal to include effects of blood draining, particularly when trying to infer on the layer-specific connections in cortex--a theme of key relevance for brain disorders like schizophrenia and for theories of brain function such as predictive coding.

Selective attentional enhancement and inhibition of fronto-posterior connectivity by the basal ganglia during attention switching

The prefrontal cortex and the basal ganglia interact to selectively gate a desired action. Recent studies have shown that this selective gating mechanism of the basal ganglia extends to the domain of attention. Here, we investigate the nature of this action-like gating mechanism for attention using a spatial attention-switching paradigm in combination with functional neuroimaging and dynamic causal modeling. We show that the basal ganglia guide attention by focally releasing inhibition of task-relevant representations, while simultaneously inhibiting task-irrelevant representations by selectively modulating prefrontal top-down connections. These results strengthen and specify the role of the basal ganglia in attention. Moreover, our findings have implications for psychological theorizing by suggesting that inhibition of unattended sensory regions is not only a consequence of mutual suppression, but is an active process, subserved by the basal ganglia.

Dopamine controls Parkinson’s tremor by inhibiting the cerebellar thalamus

Parkinsons resting tremor is related to altered cerebral activity in the basal ganglia and the cerebello-thalamo-cortical circuit. Although Parkinsons disease is characterized by dopamine depletion in the basal ganglia, the dopaminergic basis of resting tremor remains unclear: dopaminergic medication reduces tremor in some patients, but many patients have a dopamine-resistant tremor. Using pharmacological functional magnetic resonance imaging, we test how a dopaminergic intervention influences the cerebral circuit involved in Parkinsons tremor. From a sample of 40 patients with Parkinsons disease, we selected 15 patients with a clearly tremor-dominant phenotype. We compared tremor-related activity and effective connectivity (using combined electromyography-functional magnetic resonance imaging) on two occasions: ON and OFF dopaminergic medication. Building on a recently developed cerebral model of Parkinsons tremor, we tested the effect of dopamine on cerebral activity associated with the onset of tremor episodes (in the basal ganglia) and with tremor amplitude (in the cerebello-thalamo-cortical circuit). Dopaminergic medication reduced clinical resting tremor scores (mean 28%, range -12 to 68%). Furthermore, dopaminergic medication reduced tremor onset-related activity in the globus pallidus and tremor amplitude-related activity in the thalamic ventral intermediate nucleus. Network analyses using dynamic causal modelling showed that dopamine directly increased self-inhibition of the ventral intermediate nucleus, rather than indirectly influencing the cerebello-thalamo-cortical circuit through the basal ganglia. Crucially, the magnitude of thalamic self-inhibition predicted the clinical dopamine response of tremor. Dopamine reduces resting tremor by potentiating inhibitory mechanisms in a cerebellar nucleus of the thalamus (ventral intermediate nucleus). This suggests that altered dopaminergic projections to the cerebello-thalamo-cortical circuit have a role in Parkinsons tremor.aww331media15307619934001.