Dries Trippas

Logic, beliefs, and instruction: a test of the default interventionist account of belief bias.

According to dual-process accounts of thinking, belief-based responses on reasoning tasks are generated as default but can be intervened upon in favor of logical responding, given sufficient time, effort, or cognitive resource. In this article, we present the results of 5 experiments in which participants were instructed to evaluate the conclusions of logical arguments on the basis of either their logical validity or their believability. Contrary to the predictions arising from these accounts, the logical status of the presented conclusion had a greater impact on judgments concerning its believability than did the believability of the conclusion on judgments about whether it followed logically. This finding was observed when instructional set was presented as a between-participants factor (Experiment 1), when instruction was indicated prior to problem presentation by a cue (Experiment 2), and when the cue appeared simultaneously with conclusion presentation (Experiments 3 and 4). The finding also extended to a range of simple and more complex argument forms (Experiment 5). In these latter experiments, belief-based judgments took significantly longer than those made under logical instructions. We discuss the implications of these findings for default interventionist accounts of belief bias.

The SDT model of belief bias: complexity, time, and cognitive ability mediate the effects of believability.

When people evaluate conclusions, they are often influenced by prior beliefs. Prevalent theories claim that belief bias affects the quality of syllogistic reasoning. However, recent work by Dube, Rotello, and Heit (2010) has suggested that belief bias may be a simple response bias. In Experiment 1, receiver operating characteristic analysis revealed that believability affected accuracy for complex but not for simple syllogisms. In Experiment 2, the effect of believability on accuracy disappeared when judgments were made under time pressure and with participants low in cognitive capacity. The observed effects on reasoning accuracy indicate that beliefs influence more than response bias when conditions are conducive to the use of certain reasoning strategies. The findings also underscore the need to consider individual differences in reasoning.

Fluency and belief bias in deductive reasoning: new indices for old effects

Models based on signal detection theory (SDT) have occupied a prominent role in domains such as perception, categorization, and memory. Recent work by Dube et al. (2010) suggests that the framework may also offer important insights in the domain of deductive reasoning. Belief bias in reasoning has traditionally been examined using indices based on raw endorsement rates—indices that critics have claimed are highly problematic. We discuss a new set of SDT indices fit for the investigation belief bias and apply them to new data examining the effect of perceptual disfluency on belief bias in syllogisms. In contrast to the traditional approach, the SDT indices do not violate important statistical assumptions, resulting in a decreased Type 1 error rate. Based on analyses using these novel indices we demonstrate that perceptual disfluency leads to decreased reasoning accuracy, contrary to predictions. Disfluency also appears to eliminate the typical link found between cognitive ability and the effect of beliefs on accuracy. Finally, replicating previous work, we demonstrate that cognitive ability leads to an increase in reasoning accuracy and a decrease in the response bias component of belief bias.

Better but still biased: Analytic cognitive style and belief bias

Belief bias is the tendency for prior beliefs to influence peoples deductive reasoning in two ways: through the application of a simple belief-heuristic (response bias) and through the application of more effortful reasoning for unbelievable conclusions (accuracy effect or motivated reasoning). Previous research indicates that cognitive ability is the primary determinant of the effect of beliefs on accuracy. In the current study, we show that the mere tendency to engage analytic reasoning (analytic cognitive style) is responsible for the effect of cognitive ability on motivated reasoning. The implications of this finding for our understanding of the impact of individual differences on belief bias are discussed.

Using forced choice to test belief bias in syllogistic reasoning

In deductive reasoning, believable conclusions are more likely to be accepted regardless of their validity. Although many theories argue that this belief bias reflects a change in the quality of reasoning, distinguishing qualitative changes from simple response biases can be difficult (Dube, Rotello, & Heit, 2010). We introduced a novel procedure that controls for response bias. In Experiments 1 and 2, the task required judging which of two simultaneously presented syllogisms was valid. Surprisingly, there was no evidence for belief bias with this forced choice procedure. In Experiment 3, the procedure was modified so that only one set of premises was viewable at a time. An effect of beliefs emerged: unbelievable conclusions were judged more accurately, supporting the claim that beliefs affect the quality of reasoning. Experiments 4 and 5 replicated and extended this finding, showing that the effect was mediated by individual differences in cognitive ability and analytic cognitive style. Although the positive findings of Experiments 3-5 are most relevant to the debate about the mechanisms underlying belief bias, the null findings of Experiments 1 and 2 offer insight into how the presentation of an argument influences the manner in which people reason.

Logic brightens my day: evidence for implicit sensitivity to logical validity

A key assumption of dual process theory is that reasoning is an explicit, effortful, deliberative process. The present study offers evidence for an implicit, possibly intuitive component of reasoning. Participants were shown sentences embedded in logically valid or invalid arguments. Participants were not asked to reason but instead rated the sentences for liking (Experiment 1) and physical brightness (Experiments 2-3). Sentences that followed logically from preceding sentences were judged to be more likable and brighter. Two other factors thought to be linked to implicit processing-sentence believability and facial expression-had similar effects on liking and brightness ratings. The authors conclude that sensitivity to logical structure was implicit, occurring potentially automatically and outside of awareness. They discuss the results within a fluency misattribution framework and make reference to the literature on discourse comprehension. (PsycINFO Database Record

Modeling causal conditional reasoning data using SDT: caveats and new insights

In deductive reasoning, people are asked to infer the truth of an arguments conclusion given a set of premises. Research into the processes underlying deduction has focused on examining how well people discriminate between logically valid and invalid arguments, and how irrelevant factors such as ones prior beliefs interfere with the ability to reason logically (Evans et al., 1983). This normative approach to validity has traditionally informed both practice and theory in the literature. However, its critics argue that “normativism” often leads investigators to biased or misleading interpretations of phenomena (Elqayam and Evans, 2011). Formal modeling of deductive reasoning has often been successful by taking the traditional, normative approach. A case in point is the application of signal detection theory (SDT; Macmillan and Creelman, 2005) to the investigation of belief bias in syllogistic reasoning (Dube et al., 2010). In the SDT model, deductive judgments are based on strength of evidence; an argument is judged to be valid if its strength exceeds a criterion value. Because the choice of criterion is independent of the ability to discriminate between classes of arguments, the SDT model makes it possible to isolate response bias from accuracy. Dube et al. examined these two factors using ROC curves, which plot hits against false alarms at several levels of confidence. Hits and false alarms were defined in normative fashion as responding “valid” to logically valid and logically invalid conclusions, respectively. Their analysis of ROCs led them to argue two significant points. First, contrary to prevailing theories of belief bias, conclusion believability can affect response bias without affecting the quality of reasoning. Second, the curvilinear shape of the ROCs is consistent with the distributional assumptions of SDT. The latter is a key test because finding linear rather than curvilinear ROCs would be problematic for the model. The curvilinear ROCs obtained in syllogistic (see also Dube et al., 2011; Trippas et al., 2013; but see Klauer and Kellen, 2010) and other forms of reasoning (Heit and Rotello, 2010, 2014) are similar to those widely observed in memory and perception (Pazzaglia et al., 2013). This consistency across domains strengthens the case for the usefulness of the SDT approach. It also leads to an expectation of similar findings in other areas of reasoning. Below, we describe findings from conditional reasoning that violate this expectation in a surprising yet enlightening way. Causal conditionals are a form of deduction prevalent in everyday life. Consider the proposition: “If healthy foods are cheaper, then more people will eat healthy foods.” Four types of conditional inferences are possible: modus ponens (MP; “Healthy foods are cheaper, therefore more people will eat healthy foods”), modus tollens (MT; “Fewer people eat healthy foods, therefore healthy foods are not cheaper”), affirmation of the consequent (AC; “More people eat healthy foods, therefore healthy foods are cheaper”), and denial of the antecedent (DA; “Healthy foods are not cheaper, therefore less people eat healthy foods”). From a normative point of view, MP and MT are valid and AC and DA are invalid inferences. Theories differ as to how people determine validity in these problems. According to mental model theory (Johnson-Laird and Byrne, 2002), people construct an initial mental model of the conditional (e.g., p q) which may then be fleshed out by considering additional models (not-p q; not-p not-q). According to the suppositional account of the conditional (Evans et al., 2003, 2002; Evans and Over, 2004, 2012), people evaluate the subjective probability of a conditional by hypothetically supposing p and then assessing the conditional probability of q given p, P(q|p). This relation between the natural language conditional and the conditional probability, P(if p then q) = P(q|p), can be used in a Bayesian/probabilistic model of conditional inference (Oaksford et al., 2000; Oaksford and Chater, 2009, 2013). What these theories have in common is that there is no fundamental difference in how people process affirmation (MP + AC) and denial (MT + DA) inferences. This makes an SDT analysis straightforward and no different to that taken with the study of belief bias in syllogistic reasoning. For our case study, we analyzed aspects of a data set collected as part of a larger project under the direction of the fourth author of this paper1. This study examined the influence of belief in causal conditional problems (e.g., believable: “If oil prices continue to rise, then UK petrol prices will rise”; unbelievable: “If global temperatures rise, then less arctic ice will melt”). Hits were defined as “valid” responses to MP and MT and false alarms were defined as “valid” responses to AC and DA. This produced the ROCs seen in the top panel of Figure ​Figure1.1. The results are similar in some respects to those reported by Dube et al. (2010) for syllogisms: believability had no effect on accuracy (ROCs for believable and unbelievable items fall on the same curve) but seemed to affect response bias (confidence criteria for believable items are shifted to the right)2. However, there is a surprising difference: in contrast to the curvilinear ROCs observed with syllogisms, conditionals produced linear ROCs. A linear regression of the ROC (collapsing over believability) provided a good fit, R2 = 99.9%. Adding a quadratic component did not improve the fit, p = 0.78. Taken at face value, this result suggests that conditional reasoning requires a profoundly different model than the one that has seemed so successful when applied to other forms of reasoning, not to mention other cognitive tasks. Figure 1 ROC curves of causal conditionals. Top panel: Valid (MP + MT) vs. invalid (AC + DA). Bottom left: affirmation conditionals (MP vs. AC). Bottom right: denial conditionals (MT vs. DA). Points on the ROC imply a more liberal response criterion (lower confidence ... A different picture emerges when we depart from the strictly normative approach and consider separately how people respond to affirmation and denial conditionals. In the bottom left panel of Figure ​Figure1,1, plotting MP (hits) against AC (false alarms) yields typically curvilinear ROCs. Linear regression (collapsing over believability) provided a fit, R2 = 96%, that was significantly improved by the addition of a quadratic component, R2 = 99.99%, p < 0.004. Accuracy is defined by the distance of the ROCs from the chance diagonal. Contrary to the poor accuracy on display in the aggregate results in the top panel, people are quite sensitive to argument structure when affirmation is involved. In the bottom right panel of Figure ​Figure1,1, plotting MT (hits) against DA (false alarms) again yields typically curvilinear ROCs. Linear regression (collapsing over believability) provided a fit, R2 = 98%, that was significantly improved by the additional of a quadratic component, R2 = 99.99%, p < 0.002. People were sensitive to argument structure, but the position of the ROCs below the diagonal indicates that their treatment of denial arguments departed from the normative; MT are treated as less valid than AC. Applying the SDT model in a normative fashion, as would seem reasonable given extant theories of conditional reasoning, produced results that contrast sharply with previous findings. The clearly linear ROC in the top panel of Figure ​Figure11 is not only unlike the curvilinear ROCs observed with syllogisms but if taken at face value is problematic for the SDT model. It could be that there is something fundamentally different in the way that people reason about causal conditionals as compared to other types of problems. It seems to us more likely that the difference lies with affirmation and denial inferences; the latter do not seem to be treated in the normatively prescribed fashion. Once this is assumed, the ROC results become more sensible and fall in line with previous results (in a reanalysis of published and unpublished data sets, Heit and Rotello, 2014, have also reported curvilinear ROCs from MP plotted in the manner of Figure ​Figure1,1, lower left). This interpretation converges with Singmann and Klauers (2011) finding, based on state-trace analysis, that affirmation and denial problems may depend on different processes. Why use ROC analysis rather than simply examine the raw validity judgments? Interpreting the latter often relies on assumptions that may not be justified (Klauer et al., 2000; Dube et al., 2010). The main advantage of a formal model like SDT lies in its specification of assumptions. However, models can also produce insights that are not obvious at first glance. A qualitative difference between affirmation and denial inferences is not necessarily predicted by extant theories. Moreover, various manipulations seem to exert a similar effect on both types of inferences (e.g., Cummins, 1995). Finally, it is interesting to note that the production of linear ROCs when performance is driven by multiple underlying processes has been predicted in theory (DeCarlo, 2002). These results may offer a case study of how this can occur in practice.

Investigating reasoning with multiple integrated neuroscientific methods

This work was supported by the Economic and Social Research Council Grant RES- 062-23-3285. Dual processes in reasoning: A neuropsychological study of the role of working memory

Children conform, adults resist: A robot group induced peer pressure on normative social conformity

Children increasingly yielded to social pressure exerted by a group of robots; however, adults resisted being influenced by our robots. People are known to change their behavior and decisions to conform to others, even for obviously incorrect facts. Because of recent developments in artificial intelligence and robotics, robots are increasingly found in human environments, and there, they form a novel social presence. It is as yet unclear whether and to what extent these social robots are able to exert pressure similar to human peers. This study used the Asch paradigm, which shows how participants conform to others while performing a visual judgment task. We first replicated the finding that adults are influenced by their peers but showed that they resist social pressure from a group of small humanoid robots. Next, we repeated the study with 7- to 9-year-old children and showed that children conform to the robots. This raises opportunities as well as concerns for the use of social robots with young and vulnerable cross-sections of society; although conforming can be beneficial, the potential for misuse and the potential impact of erroneous performance cannot be ignored.

Characterizing belief bias in syllogistic reasoning: A hierarchical Bayesian meta-analysis of ROC data

The belief-bias effect is one of the most-studied biases in reasoning. A recent study of the phenomenon using the signal detection theory (SDT) model called into question all theoretical accounts of belief bias by demonstrating that belief-based differences in the ability to discriminate between valid and invalid syllogisms may be an artifact stemming from the use of inappropriate linear measurement models such as analysis of variance (Dube et al., Psychological Review, 117(3), 831–863, 2010). The discrepancy between Dube et al.’s, Psychological Review, 117(3), 831–863 (2010) results and the previous three decades of work, together with former’s methodological criticisms suggests the need to revisit earlier results, this time collecting confidence-rating responses. Using a hierarchical Bayesian meta-analysis, we reanalyzed a corpus of 22 confidence-rating studies (N = 993). The results indicated that extensive replications using confidence-rating data are unnecessary as the observed receiver operating characteristic functions are not systematically asymmetric. These results were subsequently corroborated by a novel experimental design based on SDT’s generalized area theorem. Although the meta-analysis confirms that believability does not influence discriminability unconditionally, it also confirmed previous results that factors such as individual differences mediate the effect. The main point is that data from previous and future studies can be safely analyzed using appropriate hierarchical methods that do not require confidence ratings. More generally, our results set a new standard for analyzing data and evaluating theories in reasoning. Important methodological and theoretical considerations for future work on belief bias and related domains are discussed.