Jelmer P. Borst

Too much control can hurt: A threaded cognition model of the attentional blink

Explanations for the attentional blink (AB; a deficit in identifying the second of two targets when presented 200-500 ms after the first) have recently shifted from limitations in memory consolidation to disruptions in cognitive control. With a new model based on the threaded cognition theory of multi-tasking we propose a different explanation: the AB is produced by an overexertion of control. This overexertion is produced by a production rule that blocks target detection during memory consolidation. In addition to fitting many known effects in the literature, the model predicts that adding certain secondary tasks will decrease the AB. In Experiment 1, a secondary task is added to the AB task in which participants have to respond to a moving dot. As predicted, AB decreases. Experiment 2 expands this result by controlling for learning, and adds a second variation, rotating the first target. For this variation the model predicts an increase in AB, which is indeed what we found.

Toward a unified theory of the multitasking continuum: from concurrent performance to task switching, interruption, and resumption

Multitasking in user behavior can be represented along a continuum in terms of the time spent on one task before switching to another. In this paper, we present a theory of behavior along the multitasking continuum, from concurrent tasks with rapid switching to sequential tasks with longer time between switching. Our theory unifies several theoretical effects - the ACT-R cognitive architecture, the threaded cognition theory of concurrent multitasking, and the memory-for-goals theory of interruption and resumption - to better understand and predict multitasking behavior. We outline the theory and discuss how it accounts for numerous phenomena in the recent empirical literature.

The Problem State: A Cognitive Bottleneck in Multitasking

The main challenge for theories of multitasking is to predict when and how tasks interfere. Here, we focus on interference related to the problem state, a directly accessible intermediate representation of the current state of a task. On the basis of Salvucci and Taatgens (2008) threaded cognition theory, we predict interference if 2 or more tasks require a problem state but not when only one task requires one. This prediction was tested in a series of 3 experiments. In Experiment 1, a subtraction task and a text entry task had to be carried out concurrently. Both tasks were presented in 2 versions: one that required maintaining a problem state and one that did not. A significant overadditive interaction effect was observed, showing that the interference between tasks was maximal when both tasks required a problem state. The other 2 experiments tested whether the interference was indeed due to a problem state bottleneck, instead of cognitive load (Experiment 2: an alternative subtraction and text entry experiment) or a phonological loop bottleneck (Experiment 3: a triple-task experiment that added phonological processing). Both experiments supported the problem state hypothesis. To account for the observed behavior, computational cognitive models were developed using threaded cognition within the context of the cognitive architecture ACT-R (Anderson, 2007). The models confirm that a problem state bottleneck can explain the observed interference.

Stroop and picture-word interference are two sides of the same coin

This article presents a cognitive model that reconciles a surprising observation in the picture—word interference (PWI) paradigm with the general notion that PWI is a form of Stroop interference. Dell’Acqua, Job, Peressotti, and Pascali (2007) assessed PWI using a psychological refractory period (PRP) paradigm, and concluded that the locus of interference in PWI is during the perceptual encoding stage. Stroop interference, on the other hand, is generally attributed to response selection. Based on these findings it was argued that PWI is not a Stroop effect. The present article discusses an alternative interpretation of these results. We assume that both effects are caused by the same interference mechanism, but that the processing speed associated with the different stimuli (colors vs. words) accounts for the previously reported differences. We support this argument by presenting a single computational model that accounts for both PWI and Stroop phenomena in single task and PRP settings.

Using model-based functional MRI to locate working memory updates and declarative memory retrievals in the fronto-parietal network

In this study, we used model-based functional MRI (fMRI) to locate two functions of the fronto-parietal network: declarative memory retrievals and updating of working memory. Because regions in the fronto-parietal network are by definition coherently active, locating functions within this network is difficult. To overcome this problem, we applied model-based fMRI, an analysis method that uses predictions of a computational model to inform the analysis. We applied model-based fMRI to five previously published datasets with associated computational cognitive models, and subsequently integrated the results in a meta-analysis. The meta-analysis showed that declarative memory retrievals correlated with activity in the inferior frontal gyrus and the anterior cingulate, whereas updating of working memory corresponded to activation in the inferior parietal lobule, as well as to activation around the inferior frontal gyrus and the anterior cingulate.

The neural correlates of problem states : Testing fMRI predictions of a computational model of multitasking

Background It has been shown that people can only maintain one problem state, or intermediate mental representation, at a time. When more than one problem state is required, for example in multitasking, performance decreases considerably. This effect has been explained in terms of a problem state bottleneck. Methodology In the current study we use the complimentary methodologies of computational cognitive modeling and neuroimaging to investigate the neural correlates of this problem state bottleneck. In particular, an existing computational cognitive model was used to generate a priori fMRI predictions for a multitasking experiment in which the problem state bottleneck plays a major role. Hemodynamic responses were predicted for five brain regions, corresponding to five cognitive resources in the model. Most importantly, we predicted the intraparietal sulcus to show a strong effect of the problem state manipulations. Conclusions Some of the predictions were confirmed by a subsequent fMRI experiment, while others were not matched by the data. The experiment supported the hypothesis that the problem state bottleneck is a plausible cause of the interference in the experiment and that it could be located in the intraparietal sulcus.

What Makes Interruptions Disruptive?: A Process-Model Account of the Effects of the Problem State Bottleneck on Task Interruption and Resumption

In this paper we present a computational cognitive model of task interruption and resumption, focusing on the effects of the problem state bottleneck. Previous studies have shown that the disruptiveness of interruptions is for an important part determined by three factors: interruption duration, interrupting-task complexity, and moment of interruption. However, an integrated theory of these effects is still missing. Based on previous research into multitasking, we propose a first step towards such a theory in the form of a process model that attributes these effects to problem state requirements of both the interrupted and the interrupting task. Subsequently, we tested two predictions of this model in two experiments. The experiments confirmed that problem state requirements are an important predictor for the disruptiveness of interruptions. This suggests that interfaces should be designed to a) interrupt users at low-problem state moments and b) maintain the problem state for the user when interrupted.

Pupil Dilation Co-Varies with Memory Strength of Individual Traces in a Delayed Response Paired-Associate Task

Studies on cognitive effort have shown that pupil dilation is a reliable indicator of memory load. However, it is conceivable that there are other sources of effort involved in memory that also affect pupil dilation. One of these is the ease with which an item can be retrieved from memory. Here, we present the results of an experiment in which we studied the way in which pupil dilation acts as an online marker for memory processing during the retrieval of paired associates while reducing confounds associated with motor responses. Paired associates were categorized into sets containing either 4 or 7 items. After learning the paired associates once, pupil dilation was measured during the presentation of the retrieval cue during four repetitions of each set. Memory strength was operationalized as the number of repetitions (frequency) and set-size, since having more items per set results in a lower average recency. Dilation decreased with increased memory strength, supporting the hypothesis that the amplitude of the evoked pupillary response correlates positively with retrieval effort. Thus, while many studies have shown that “memory load” influences pupil dilation, our results indicate that the task-evoked pupillary response is also sensitive to the experimentally manipulated memory strength of individual items. As these effects were observed well before the response had been given, this study also suggests that pupil dilation can be used to assess an item’s memory strength without requiring an overt response.

The Discovery of Processing Stages: Analyzing EEG data with Hidden Semi-Markov Models

In this paper we propose a new method for identifying processing stages in human information processing. Since the 1860s scientists have used different methods to identify processing stages, usually based on reaction time (RT) differences between conditions. To overcome the limitations of RT-based methods we used hidden semi-Markov models (HSMMs) to analyze EEG data. This HSMM-EEG methodology can identify stages of processing and how they vary with experimental condition. By combining this information with the brain signatures of the identified stages one can infer their function, and deduce underlying cognitive processes. To demonstrate the method we applied it to an associative recognition task. The stage-discovery method indicated that three major processes play a role in associative recognition: a familiarity process, an associative retrieval process, and a decision process. We conclude that the new stage-discovery method can provide valuable insight into human information processing.

What happens when we switch tasks: pupil dilation in multitasking

Interruption studies typically focus on external interruptions, even though self-interruptions occur at least as often in real work environments. In this article, we therefore contrast external interruptions with self-interruptions. Three multitasking experiments were conducted, in which we examined changes in pupil size when participants switched from a primary to a secondary task. Results showed an increase in pupil dilation several seconds before a self-interruption, which we could attribute to the decision to switch. This indicates that the decision takes a relatively large amount of time. This was supported by the fact that in Experiment 2, participants were significantly slower on the self-interruption blocks than on the external interruption blocks. These findings suggest that the decision to switch is costly, but may also be open for modification through appropriate training. In addition, we propose that if one must switch tasks, it can be more efficient to implement a forced switch after the completion of a subtask instead of leaving the decision to the user.