David K. Sewell

Predicting Perceptual Decision Biases from Early Brain Activity

Perceptual decision making is believed to be driven by the accumulation of sensory evidence following stimulus encoding. More controversially, some studies report that neural activity preceding the stimulus also affects the decision process. We used a multivariate pattern classification approach for the analysis of the human electroencephalogram (EEG) to decode choice outcomes in a perceptual decision task from spatially and temporally distributed patterns of brain signals. When stimuli provided discriminative information, choice outcomes were predicted by neural activity following stimulus encoding; when stimuli provided no discriminative information, choice outcomes were predicted by neural activity preceding the stimulus. Moreover, in the absence of discriminative information, the recent choice history primed the choices on subsequent trials. A diffusion model fitted to the choice probabilities and response time distributions showed that the starting point of the evidence accumulation process was shifted toward the previous choice, consistent with the hypothesis that choice priming biases the accumulation process toward a decision boundary. This bias is reflected in prestimulus brain activity, which, in turn, becomes predictive of future decisions. Our results provide a model of how non-stimulus-driven decision making in humans could be accomplished on a neural level.

A competitive interaction theory of attentional selection and decision making in brief, multielement displays.

We generalize the integrated system model of Smith and Ratcliff (2009) to obtain a new theory of attentional selection in brief, multielement visual displays. The theory proposes that attentional selection occurs via competitive interactions among detectors that signal the presence of task-relevant features at particular display locations. The outcome of the competition, together with attention, determines which stimuli are selected into visual short-term memory (VSTM). Decisions about the contents of VSTM are made by a diffusion-process decision stage. The selection process is modeled by coupled systems of shunting equations, which perform gated where-on-what pathway VSTM selection. The theory provides a computational account of key findings from attention tasks with near-threshold stimuli. These are (a) the success of the MAX model of visual search and spatial cuing, (b) the distractor homogeneity effect, (c) the double-target detection deficit, (d) redundancy costs in the post-stimulus probe task, (e) the joint item and information capacity limits of VSTM, and (f) the object-based nature of attentional selection. We argue that these phenomena are all manifestations of an underlying competitive VSTM selection process, which arise as a natural consequence of our theory.

Cued detection with compound integration-interruption masks reveals multiple attentional mechanisms

The relationship between attention and visual masking was investigated in a cued detection task using a factorial masking manipulation. Stimuli were either unmasked, or were masked with simultaneous (integration) masks, or delayed (interruption) masks, or integration-interruption mask pairs. The cuing effects in detection sensitivity were smallest with unmasked stimuli, intermediate with single masks, and largest with integration-interruption pairs. Large cuing effects in RT were found in all stimulus conditions. The results are inconsistent with general mechanisms of contrast gain and response gain, which do not predict interactions with interruption masks. The data were modeled using the integrated system model of visual attention of P. L. Smith and R. Ratcliff (2009), which provides an account of both RT and accuracy. The model fits suggest the action of two independent attentional mechanisms: an early selection mechanism that enhances the perceptual representation of attended, noisy stimuli, and a late selection mechanism that increases the rate of information transfer to visual short-term memory. The results are consistent with a distributed, multi-locus system of attentional control.

Attention and Working Memory Capacity: Insights from Blocking, Highlighting, and Knowledge Restructuring.

The concept of attention is central to theorizing in learning as well as in working memory. However, research to date has yet to establish how attention as construed in one domain maps onto the other. We investigate two manifestations of attention in category- and cue-learning to examine whether they might provide common ground between learning and working memory. Experiment 1 examined blocking and highlighting effects in an associative learning paradigm, which are widely thought to be attentionally mediated. No relationship between attentional performance indicators and working memory capacity (WMC) was observed, despite the fact that WMC was strongly associated with overall learning performance. Experiment 2 used a knowledge restructuring paradigm, which is known to require recoordination of partial category knowledge using representational attention. We found that the extent to which people successfully recoordinated their knowledge was related to WMC. The results illustrate a link between WMC and representational-but not dimensional-attention in category learning.

Restructuring partitioned knowledge: The role of recoordination in category learning

Knowledge restructuring refers to changes in the strategy with which people solve a given problem. Two types of knowledge restructuring are supported by existing category learning models. The first is a relearning process, which involves incremental updating of knowledge as learning progresses. The second is a recoordination process, which involves novel changes in the way existing knowledge is applied to the task. Whereas relearning is supported by both single- and multiple-module models of category learning, only multiple-module models support recoordination. To date, only relearning has been directly supported empirically. We report two category learning experiments that provide direct evidence of recoordination. People can fluidly alternate between different categorization strategies, and moreover, can reinstate an old strategy even after prolonged use of an alternative. The knowledge restructuring data are not well fit by a single-module model (ALCOVE). By contrast, a multiple-module model (ATRIUM) quantitatively accounts for recoordination. Low-level changes in the distribution of dimensional attention are shown to subsequently affect how ATRIUM coordinates its modular knowledge. We argue that learning about complex tasks occurs at the level of the partial knowledge elements used to generate a response strategy.

Attentional control in visual signal detection: effects of abrupt-onset and no-onset stimuli

The attention literature distinguishes two general mechanisms by which attention can benefit performance: gain (or resource) models and orienting (or switching) models. In gain models, processing efficiency is a function of a spatial distribution of capacity or resources; in orienting models, an attentional spotlight must be aligned with the stimulus location, and processing efficiency is a function of when this occurs. Although they involve different processing mechanisms, these models are difficult to distinguish empirically. We compared performance with abrupt-onset and no-onset Gabor patch stimuli in a cued detection task in which we obtained distributions of reaction time (RT) and accuracy as a function of stimulus contrast. In comparison to abrupt-onset stimuli, RTs to miscued no-onset stimuli were increased and accuracy was reduced. Modeling the data with the integrated system model of Philip L. Smith and Roger Ratcliff (2009) provided evidence for reallocation of processing resources during the course of a trial, consistent with an orienting account. Our results support a view of attention in which processing efficiency depends on a dynamic spatiotemporal distribution of resources that has both gain and orienting properties.

The attention-weighted sample-size model of visual short-term memory: Attention capture predicts resource allocation and memory load.

We investigated the capacity of visual short-term memory (VSTM) in a phase discrimination task that required judgments about the configural relations between pairs of black and white features. Sewell et al. (2014) previously showed that VSTM capacity in an orientation discrimination task was well described by a sample-size model, which views VSTM as a resource comprised of a finite number of noisy stimulus samples. The model predicts the invariance of [Formula: see text] , the sum of squared sensitivities across items, for displays of different sizes. For phase discrimination, the set-size effect significantly exceeded that predicted by the sample-size model for both simultaneously and sequentially presented stimuli. Instead, the set-size effect and the serial position curves with sequential presentation were predicted by an attention-weighted version of the sample-size model, which assumes that one of the items in the display captures attention and receives a disproportionate share of resources. The choice probabilities and response time distributions from the task were well described by a diffusion decision model in which the drift rates embodied the assumptions of the attention-weighted sample-size model.

From shunting inhibition to dynamic normalization: attentional selection and decision-making in brief visual displays

Normalization models of visual sensitivity assume that the response of a visual mechanism is scaled divisively by the sum of the activity in the excitatory and inhibitory mechanisms in its neighborhood. Normalization models of attention assume that the weighting of excitatory and inhibitory mechanisms is modulated by attention. Such models have provided explanations of the effects of attention in both behavioral and single-cell recording studies. We show how normalization models can be obtained as the asymptotic solutions of shunting differential equations, in which stimulus inputs and the activity in the mechanism control growth rates multiplicatively rather than additively. The value of the shunting equation approach is that it characterizes the entire time course of the response, not just its asymptotic strength. We describe two models of attention based on shunting dynamics, the integrated system model of Smith and Ratcliff (2009) and the competitive interaction theory of Smith and Sewell (2013). These models assume that attention, stimulus salience, and the observers strategy for the task jointly determine the selection of stimuli into visual short-term memory (VSTM) and the way in which stimulus representations are weighted. The quality of the VSTM representation determines the speed and accuracy of the decision. The models provide a unified account of a variety of attentional phenomena found in psychophysical tasks using single-element and multi-element displays. Our results show the generality and utility of the normalization approach to modeling attention.

Response time modeling reveals multiple contextual cuing mechanisms

Contextual cuing refers to a response time (RT) benefit that occurs when observers search through displays that have been repeated over the course of an experiment. Although it is generally agreed that contextual cuing arises via an associative learning mechanism, there is uncertainty about the type(s) of process(es) that allow learning to influence RT. We contrast two leading accounts of the contextual cuing effect that differ in terms of the general process that is credited with producing the effect. The first, the expedited search account, attributes the cuing effect to an increase in the speed with which the target is acquired. The second, the decision threshold account, attributes the cuing effect to a reduction in the response threshold used by observers when making a subsequent decision about the target (e.g., judging its orientation). We use the diffusion model to contrast the quantitative predictions of these two accounts at the level of individual observers. Our use of the diffusion model allows us to also explore a novel decision-level locus of the cuing effect based on perceptual learning. This novel account attributes the RT benefit to a perceptual learning process that increases the quality of information used to drive the decision process. Our results reveal both individual differences in the process(es) involved in contextual cuing but also identify several striking regularities across observers. We find strong support for both the decision threshold account as well as the novel perceptual learning account. We find relatively weak support for the expedited search account.

The psychology and psychobiology of simple decisions: speeded choice and its neural correlates

In this chapter, we provide a tutorial review of the class of sequential sampling models of two-choice decision-making. These models, which have been developed in cognitive and mathematical psychology over the last 50 years, provide a detailed quantitative account of performance in simple, speeded choice tasks. The models explain the major findings from a wide variety of behavioral decision tasks, including the relationship between choice probabilities and response time (RT), the speed-accuracy tradeoff, the shapes of RT distributions, and the relative speed of correct and error responses. More recently, electrophysiological recordings from decision-related brain areas in awake behaving monkeys have revealed a correspondence between patterns of neural firing and the statistical processes of evidence accumulation assumed in the psychological models. We discuss the theoretical relationship between the cognitive process of evidence accumulation and neural firing rates and show how neural data can constrain behavioral models. Importantly, constraints from neurophysiological data can be used to test between models that are otherwise difficult to distinguish. The convergence of psychological theory and neurophysiological data suggests that a common theoretical and mathematical framework is sufficient to account for simple decision-making data at neural and behavioral levels of analysis.