Maggie E. Toplak

The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks.

The Cognitive Reflection Test (CRT; Frederick, 2005) is designed to measure the tendency to override a prepotent response alternative that is incorrect and to engage in further reflection that leads to the correct response. In this study, we showed that the CRT is a more potent predictor of performance on a wide sample of tasks from the heuristics-and-biases literature than measures of cognitive ability, thinking dispositions, and executive functioning. Although the CRT has a substantial correlation with cognitive ability, a series of regression analyses indicated that the CRT was a unique predictor of performance on heuristics-and-biases tasks. It accounted for substantial additional variance after the other measures of individual differences had been statistically controlled. We conjecture that this is because neither intelligence tests nor measures of executive functioning assess the tendency toward miserly processing in the way that the CRT does. We argue that the CRT is a particularly potent measure of the tendency toward miserly processing because it is a performance measure rather than a self-report measure.

Practitioner Review: Do performance-based measures and ratings of executive function assess the same construct?

BACKGROUND Both performance-based and rating measures are commonly used to index executive function in clinical and neuropsychological assessments. They are intended to index the same broad underlying mental construct of executive function. The association between these two types of measures was investigated in the current article. METHOD AND RESULTS We examined the association between performance-based and rating measures of executive function in 20 studies. These studies included 13 child and 7 adult samples, which were derived from 7 clinical, 2 nonclinical, and 11 combined clinical and nonclinical samples. Only 68 (24%) of the 286 relevant correlations reported in these studies were statistically significant, and the overall median correlation was only .19. CONCLUSIONS It was concluded that performance-based and rating measures of executive function assess different underlying mental constructs. We discuss how these two types of measures appear to capture different levels of cognition, namely, the efficiency of cognitive abilities and success in goal pursuit. Clinical implications of using performance-based and rating measures of executive function are discussed, including the use of these measures in assessing ADHD.

Do performance‐based measures and ratings of executive function assess the same construct?

BACKGROUND Both performance-based and rating measures are commonly used to index executive function in clinical and neuropsychological assessments. They are intended to index the same broad underlying mental construct of executive function. The association between these two types of measures was investigated in the current article. METHOD AND RESULTS We examined the association between performance-based and rating measures of executive function in 20 studies. These studies included 13 child and 7 adult samples, which were derived from 7 clinical, 2 nonclinical, and 11 combined clinical and nonclinical samples. Only 68 (24%) of the 286 relevant correlations reported in these studies were statistically significant, and the overall median correlation was only .19. CONCLUSIONS It was concluded that performance-based and rating measures of executive function assess different underlying mental constructs. We discuss how these two types of measures appear to capture different levels of cognition, namely, the efficiency of cognitive abilities and success in goal pursuit. Clinical implications of using performance-based and rating measures of executive function are discussed, including the use of these measures in assessing ADHD.

Executive Functions: Performance-Based Measures and the Behavior Rating Inventory of Executive Function (BRIEF) in Adolescents with Attention Deficit/Hyperactivity Disorder (ADHD)

Performance-based measures and ratings of executive functions were examined in a sample of adolescents with attention deficit/hyperactivity disorder (ADHD) and comparison controls. Performance-based measures of executive function included inhibition, working memory, set shifting, and planning, and ratings of these same executive functions were completed by parents and teachers. Adolescents with ADHD demonstrated lower executive function performance than controls and displayed elevated ratings on the executive function ratings by parents and teachers. Significant associations were obtained between the performance-based measures and the parent and teacher ratings, but each measure was not uniquely associated with its respective scale on the rating scales. When performance-based measures and ratings were examined as predictors of ADHD status, the parent and teacher ratings entered as significant predictors of ADHD status. Further commonality analyses indicated that performance-based measures accounted for little unique variance in predicting ADHD status and also displayed little overlap with the behavioral ratings. These findings highlight the diagnostic utility of behavioral ratings of executive function in predicting ADHD status; however, behavioral ratings should not be assumed to be a proxy for performance on measures of executive function in clinical practice.

Temporal information processing in ADHD : Findings to date and new methods

The ability to perceive and represent time is a fundamental but complex cognitive skill that allows us to perceive and organize sequences of events and actions, and to anticipate or predict when future events will occur. It is a multidimensional construct, and a variety of methods have been used to understand timing performance in ADHD samples, which makes it difficult to integrate findings across studies. While further replication is needed, growing evidence links ADHD to problems in several aspects of temporal information processing, including duration discrimination, duration reproduction, and finger tapping. Neuroimaging studies of ADHD have also implicated cerebellar, basal ganglia, and prefrontal regions of the brain, which are believed to subserve temporal information processing. This line of research implicates more basic cognitive mechanisms than previously linked with ADHD and challenges researchers to develop and utilize innovative, multidisciplinary, scientific methods to dissect the various components of temporal information processing. Recent advances in neuroimaging, such as magnetoencephalography in collaboration with structural magnetic resonance imaging, can discriminate temporal processing at the level of a millisecond. This approach can lay the groundwork to provide a more precise understanding of neural network activity during different aspects and stages of temporal information processing in ADHD.

Executive and motivational processes in adolescents with Attention-Deficit-Hyperactivity Disorder (ADHD)

BackgroundThe objective of the current study was to examine performance and correlates of performance on a decision-making card task involving risky choices (Iowa Gambling Task) in adolescents with ADHD and comparison controls. Forty-four participants with ADHD and 34 controls were administered measures of estimated intellectual ability, working memory, and the card task. Also, behavioural ratings were obtained from parents and teachers.ResultsAdolescents with ADHD scored lower on the measures of intellectual ability, working memory, and made less advantageous selections on the card task compared to controls. Performance on measures of intellectual ability and working memory were unrelated to card task performance in both the ADHD and control samples. Parent ratings of hyperactivity/impulsivity were significantly associated with card task performance in the adolescents with ADHD, but not in controls.ConclusionThese findings demonstrate impaired decision-making in adolescents with ADHD, and the separability of motivational and executive function processes, supporting current dual pathway models of ADHD.

Decision-making and cognitive abilities: A review of associations between Iowa Gambling Task performance, executive functions, and intelligence.

The Iowa Gambling Task (IGT) has been used to study decision-making differences in many different clinical and developmental samples. It has been suggested that IGT performance captures abilities that are separable from cognitive abilities, including executive functions and intelligence. The purpose of the current review was to examine studies that have explicitly examined the relationship between IGT performance and these cognitive abilities. We included 43 studies that reported correlational analyses with IGT performance, including measures of inhibition, working memory, and set-shifting as indices of executive functions, as well as measures of verbal, nonverbal, and full-scale IQ as indices of intelligence. Overall, only a small proportion of the studies reported a statistically significant relationship between IGT performance and these cognitive abilities. The majority of studies reported a non-significant relationship. Of the minority of studies that reported statistically significant effects, effect sizes were, at best, small to modest, and confidence intervals were large, indicating that considerable variability in performance on the IGT is not captured by current measures of executive function and intelligence. These findings highlight the separability between decision-making on the IGT and cognitive abilities, which is consistent with recent conceptualizations that differentiate rationality from intelligence.

Heuristic and Analytic Processing: Age Trends and Associations with Cognitive Ability and Cognitive Styles.

Developmental and individual differences in the tendency to favor analytic responses over heuristic responses were examined in children of two different ages (10- and 11-year-olds versus 13-year-olds), and of widely varying cognitive ability. Three tasks were examined that all required analytic processing to override heuristic processing: inductive reasoning, deductive reasoning under conditions of belief bias, and probabilistic reasoning. Significant increases in analytic responding with development were observed on the first two tasks. Cognitive ability was associated with analytic responding on all three tasks. Cognitive style measures such as actively open-minded thinking and need for cognition explained variance in analytic responding on the tasks after variance shared with cognitive ability had been controlled. The implications for dual-process theories of cognition and cognitive development are discussed.

Time perception deficits in attention-deficit/ hyperactivity disorder and comorbid reading difficulties in child and adolescent samples.

BACKGROUND Our objective was to investigate time perception in Attention-Deficit/Hyperactivity Disorder (ADHD) with and without comorbid reading difficulties (RD) in child and adolescent participants. METHOD In study 1, 50 children with ADHD (31 ADHD, 19 ADHD+RD) and age-matched healthy controls (n = 50) completed three psychophysical tasks: duration discrimination (target duration of 400 ms versus a foil duration), frequency discrimination (a control condition to evaluate general perceptual ability), and a duration estimation task using the method of reproduction for intervals of 400 ms, 2000 ms, and 6000 ms. Study 2 used the same tasks with an adolescent sample (35 ADHD, 24 ADHD+RD, 39 controls). RESULTS In both studies, children and adolescents with ADHD and ADHD+RD displayed some impairments in duration discrimination and the precision with which they reproduced the intervals on the estimation task, particularly the shorter 400 ms interval. The most severe impairments tended to occur in the comorbid ADHD+RD group. No impairments were found on the frequency discrimination task. ADHD participants also displayed significant intra-individual variability in their performance on the estimation task. Finally, short-term and working memory, estimated full-scale IQ, and teacher report of hyperactivity/impulsivity were found to differentially predict performance on the time perception measures in the adolescent clinical sample. CONCLUSIONS Deficits in duration discrimination, duration estimation, and intra-individual performance variability may have cascaded effects on the temporal organisation of behaviour in children and adolescents with ADHD and ADHD+RD.

Assessing miserly information processing: An expansion of the Cognitive Reflection Test

The Cognitive Reflection Test (CRT; Frederick, 2005) is designed to measure the tendency to override a prepotent response alternative that is incorrect and to engage in further reflection that leads to the correct response. It is a prime measure of the miserly information processing posited by most dual process theories. The original three-item test may be becoming known to potential participants, however. We examined a four-item version that could serve as a substitute for the original. Our data show that it displays a .58 correlation with the original version and that it has very similar relationships with cognitive ability, various thinking dispositions, and with several other rational thinking tasks. Combining the two versions into a seven-item test resulted in a measure of miserly processing with substantial reliability (.72). The seven-item version was a strong independent predictor of performance on rational thinking tasks after the variance accounted for by cognitive ability and thinking dispositions had been partialled out.