Rafal Bogacz

The Physics of Optimal Decision Making: A Formal Analysis of Models of Performance in Two-Alternative Forced-Choice Tasks.

In this article, the authors consider optimal decision making in two-alternative forced-choice (TAFC) tasks. They begin by analyzing 6 models of TAFC decision making and show that all but one can be reduced to the drift diffusion model, implementing the statistically optimal algorithm (most accurate for a given speed or fastest for a given accuracy). They prove further that there is always an optimal trade-off between speed and accuracy that maximizes various reward functions, including reward rate (percentage of correct responses per unit time), as well as several other objective functions, including ones weighted for accuracy. They use these findings to address empirical data and make novel predictions about performance under optimality.

The neural basis of the speed–accuracy tradeoff

In many situations, decision makers need to negotiate between the competing demands of response speed and response accuracy, a dilemma generally known as the speed-accuracy tradeoff (SAT). Despite the ubiquity of SAT, the question of how neural decision circuits implement SAT has received little attention up until a year ago. We review recent studies that show SAT is modulated in association and pre-motor areas rather than in sensory or primary motor areas. Furthermore, the studies suggest that emphasis on response speed increases the baseline firing rate of cortical integrator neurons. We also review current theories on how and where in the brain the SAT is controlled, and we end by proposing research directions that could distinguish between these theories.

The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions

Neurophysiological studies have identified a number of brain regions critically involved in solving the problem of action selection or decision making. In the case of highly practiced tasks, these regions include cortical areas hypothesized to integrate evidence supporting alternative actions and the basal ganglia, hypothesized to act as a central switch in gating behavioral requests. However, despite our relatively detailed knowledge of basal ganglia biology and its connectivity with the cortex and numerical simulation studies demonstrating selective function, no formal theoretical framework exists that supplies an algorithmic description of these circuits. This article shows how many aspects of the anatomy and physiology of the circuit involving the cortex and basal ganglia are exactly those required to implement the computation defined by an asymptotically optimal statistical test for decision making: the multihypothesis sequential probability ratio test (MSPRT). The resulting model of basal ganglia provides a new framework for understanding the computation in the basal ganglia during decision making in highly practiced tasks. The predictions of the theory concerning the properties of particular neuronal populations are validated in existing experimental data. Further, we show that this neurobiologically grounded implementation of MSPRT outperforms other candidates for neural decision making, that it is structurally and parametrically robust, and that it can accommodate cortical mechanisms for decision making in a way that complements those in basal ganglia.

Cortico-striatal connections predict control over speed and accuracy in perceptual decision making

When people make decisions they often face opposing demands for response speed and response accuracy, a process likely mediated by response thresholds. According to the striatal hypothesis, people decrease response thresholds by increasing activation from cortex to striatum, releasing the brain from inhibition. According to the STN hypothesis, people decrease response thresholds by decreasing activation from cortex to subthalamic nucleus (STN); a decrease in STN activity is likewise thought to release the brain from inhibition and result in responses that are fast but error-prone. To test these hypotheses—both of which may be true—we conducted two experiments on perceptual decision making in which we used cues to vary the demands for speed vs. accuracy. In both experiments, behavioral data and mathematical model analyses confirmed that instruction from the cue selectively affected the setting of response thresholds. In the first experiment we used ultra-high-resolution 7T structural MRI to locate the STN precisely. We then used 3T structural MRI and probabilistic tractography to quantify the connectivity between the relevant brain areas. The results showed that participants who flexibly change response thresholds (as quantified by the mathematical model) have strong structural connections between presupplementary motor area and striatum. This result was confirmed in an independent second experiment. In general, these findings show that individual differences in elementary cognitive tasks are partly driven by structural differences in brain connectivity. Specifically, these findings support a cortico-striatal control account of how the brain implements adaptive switches between cautious and risky behavior.

Optimal decision-making theories: linking neurobiology with behaviour.

This article reviews recently proposed theories postulating that, during simple choices, the brain performs statistically optimal decision making. These theories are ecologically motivated by evolutionary pressures to optimize the speed and accuracy of decisions and to maximize the rate of receiving rewards for correct choices. This article suggests that the models of decision making that are proposed on different levels of abstraction can be linked by virtue of the same optimal computation. Also reviewed here are recent observations that many aspects of the circuit that involves the cortex and basal ganglia are the same as those that are required to perform statistically optimal choice. This review illustrates how optimal-decision theories elucidate current data and provide experimental predictions that concern both neurobiology and behaviour.

Conditions for the generation of beta oscillations in the subthalamic nucleus-globus pallidus network

The advance of Parkinsons disease is associated with the existence of abnormal oscillations within the basal ganglia with frequencies in the beta band (13–30 Hz). While the origin of these oscillations remains unknown, there is some evidence suggesting that oscillations observed in the basal ganglia arise due to interactions of two nuclei: the subthalamic nucleus (STN) and the globus pallidus pars externa (GPe). To investigate this hypothesis, we develop a computational model of the STN–GPe network based upon anatomical and electrophysiological studies. Significantly, our study shows that for certain parameter regimes, the model intrinsically oscillates in the beta range. Through an analytical study of the model, we identify a simple set of necessary conditions on model parameters that guarantees the existence of beta oscillations. These conditions for generation of oscillations are described by a set of simple inequalities and can be summarized as follows: (1) The excitatory connections from STN to GPe and the inhibitory connections from GPe to STN need to be sufficiently strong. (2) The time required by neurons to react to their inputs needs to be short relative to synaptic transmission delays. (3) The excitatory input from the cortex to STN needs to be high relative to the inhibition from striatum to GPe. We confirmed the validity of these conditions via numerical simulation. These conditions describe changes in parameters that are consistent with those expected as a result of the development of Parkinsons disease, and predict manipulations that could inhibit the pathological oscillations.

On optimal decision-making in brains and social insect colonies

The problem of how to compromise between speed and accuracy in decision-making faces organisms at many levels of biological complexity. Striking parallels are evident between decision-making in primate brains and collective decision-making in social insect colonies: in both systems, separate populations accumulate evidence for alternative choices; when one population reaches a threshold, a decision is made for the corresponding alternative, and this threshold may be varied to compromise between the speed and the accuracy of decision-making. In primate decision-making, simple models of these processes have been shown, under certain parametrizations, to implement the statistically optimal procedure that minimizes decision time for any given error rate. In this paper, we adapt these same analysis techniques and apply them to new models of collective decision-making in social insect colonies. We show that social insect colonies may also be able to achieve statistically optimal collective decision-making in a very similar way to primate brains, via direct competition between evidence-accumulating populations. This optimality result makes testable predictions for how collective decision-making in social insects should be organized. Our approach also represents the first attempt to identify a common theoretical framework for the study of decision-making in diverse biological systems.

Extending a biologically inspired model of choice: multi-alternatives, nonlinearity and value-based multidimensional choice.

The leaky competing accumulator (LCA) is a biologically inspired model of choice. It describes the processes of leaky accumulation and competition observed in neuronal populations during choice tasks and it accounts for reaction time distributions observed in psychophysical experiments. This paper discusses recent analyses and extensions of the LCA model. First, it reviews the dynamics and examines the conditions that make the model achieve optimal performance. Second, it shows that nonlinearities of the type present in biological neurons improve performance when the number of choice alternatives increases. Third, the model is extended to value-based choice, where it is shown that nonlinearities in the value function explain risk aversion in risky choice and preference reversals in choice between alternatives characterized across multiple dimensions.

SIMPLE NEURAL NETWORKS THAT OPTIMIZE DECISIONS

We review simple connectionist and firing rate models for mutually inhibiting pools of neurons that discriminate between pairs of stimuli. Both are two-dimensional nonlinear stochastic ordinary differential equations, and although they differ in how inputs and stimuli enter, we show that they are equivalent under state variable and parameter coordinate changes. A key parameter is gain: the maximum slope of the sigmoidal activation function. We develop piecewise-linear and purely linear models, and one-dimensional reductions to Ornstein–Uhlenbeck processes that can be viewed as linear filters, and show that reaction time and error rate statistics are well approximated by these simpler models. We then pose and solve the optimal gain problem for the Ornstein–Uhlenbeck processes, finding explicit gain schedules that minimize error rates for time-varying stimuli. We relate these to time courses of norepinephrine release in cortical areas, and argue that transient firing rate changes in the brainstem nucleus locus coeruleus may be responsible for approximate gain optimization.

Model of Familiarity Discrimination in the Perirhinal Cortex

Much evidence indicates that recognition memory involves two separable processes, recollection and familiarity discrimination, with familiarity discrimination being dependent on the perirhinal cortex of the temporal lobe. Here, we describe a new neural network model designed to mimic the response patterns of perirhinal neurons that signal information concerning the novelty or familiarity of stimuli. The model achieves very fast and accurate familiarity discrimination while employing biologically plausible parameters and Hebbian learning rules. The fact that the activity patterns of the models simulated neurons are closely similar to those of neurons recorded from the primate perirhinal cortex indicates that this brain region could discriminate familiarity using principles akin to those of the model. If so, the capacity of the model establishes that the perirhinal cortex alone may discriminate the familiarity of many more stimuli than current neural network models indicate could be recalled (recollected) by all the remaining areas of the cerebral cortex. This efficiency and speed of detecting novelty provides an evolutionary advantage, thereby providing a reason for the existence of a familiarity discrimination network in addition to networks used for recollection.