Gideon Keren

Two Is Not Always Better Than One: A Critical Evaluation of Two-System Theories

Over the past two decades, there has been an upsurge in theoretical frameworks alluding to the existence of two different processing systems that supposedly operate according to different rules. This article critically examines the scientific advance offered by these theories (in particular advances in the domains of reasoning, decision making, and social cognition) and questions their theoretical coherence as well as the evidence for their existence. We scrutinize the conceptual underpinnings of two-system models and explicate the assumptions underlying these models to see whether they are reasonable. We also evaluate the empirical paradigms used to validate two-system models and ponder about their explanatory strength and predictive power. Given the popularity of these models, we discuss the appeal of two-system theories and suggest potential reasons for their prevalence. We comment on the potential costs associated with these models and allude to the desired nature of potential alternatives. We conclude that two-system models currently provide little scientific advance, and we encourage researchers to adopt more rigorous conceptual definitions and employ more stringent criteria for testing the empirical evidence in support for two-system theories.

Islanders and hostages: Deep and surface structures of decision problems☆

Abstract Decision problems can be characterized by their surface structure (the story as presented) and their deep structure (the representation of the problem used by a respondent or as specified by a psychological theory). We started with a story problem about an island with a life-threatening epidemic. The authors of this problem (Hammerton, Jones-Lee and Abbott 1982) assumed a deep structure based on subjectively expected utility (SEU) theory and concluded that a large majority of their respondents were risk averse. We developed eleven variations of the surface structure, all of which could be represented by the same deep structure. The responses of 1,366 Dutch, American, and Israeli subjects varied widely as a function of some of these surface changes, indicating that underlying deep structure was not at all like the simple SEU representation. The translation from surface structure to deep structure can be quite complex; researches should not make untested assumptions about this translation when using story problems to test their theories.

A tale of two systems : A scientific advance or a theoretical stone soup? Commentary on Evans & Stanovich (2013)

In a Russian folktale, a scamp teaches a fool how to prepare a delicious soup that requires nothing but a soup stone. All one has to do is put the soup stone into boiling water. Yet, to achieve a better taste, it is desirable to add some vegetables, some meat to make the soup especially tasty, and some salt and pepper will do no harm, and so forth. A methodical examination of the development of two-system models during the past 10 years is reminiscent of the above folktale. Indeed, inspecting the different labels proposed and the various terminologies employed to characterize the presumed two systems and their corresponding alleged processes strongly suggest that it has become a stone soup where everything goes. Several authors (e.g., Gigerenzer & Reiger, 1996; Keren & Schul, 2009; Kruglanski & Gigerenzer, 2011; Osman, 2004) have critically assessed the viability and usefulness of the two-system approach. The recent expositions by Evans and Stanovich (2013, this issue, henceforth E&S) in this issue and elsewhere (e.g., Evans, 2009, 2011; Stanovich, 2009) suggest that they recognize many of the problems and seek to rectify them. Unfortunately, I suggest that their attempts are futile and, if anything, only increase confusion due to the addition of new theoretical constructs and the proposition that there may be more than two systems. My commentary has two different levels. At one level, I directly address E&S’s article, particularly commenting on what they now term a two-process theory. At a higher level, I consider E&S’s new model to be yet another version of what generically is referred to as “two-system” models and ask whether these indeed constitute theories or models as usually conceived by the scientific community. It is obviously beyond the scope of this article to provide an elaborated discussion of the nature of adequate scientific theories. Yet, it is commonly agreed that proper models should be based on well-defined constructs, motivate new questions, and yield testable predictions that will either corroborate or refute the theory under investigation. The central claim of this commentary is that none of the two-system theories, including the current version from E&S, satisfy the cardinal requirements of a scientific theory. Indeed, a previous article written in collaboration with my colleague Yaacov Schul (Keren & Schul, 2009) was meant to specify some (but certainly not all) basic requirements needed to transform two-system theories into testable ones. We realized that satisfying these requirements may be difficult, if not impossible, because these theories are so broad that they supposedly can account for almost all high-level social-cognitive phenomena. An important feature, in fact a necessary condition, for any scientific theory is a clear and unambiguous statement regarding the theory’s constraints and the corresponding boundaries. Otherwise, the theory becomes nothing but a set of arbitrary nominal definitions as the testable empirical world is not constrained. Inspection of two-system theories, including E&S’s version currently under discussion, unequivocally shows that the selective evidence they state to support their claims is, at best, consistent with the theoretical position rather than being based on direct tests derived from their theories. E&S’s article boils down to three main points: First, they agree with many of the criticisms of two-system models, yet believe that these were overstated. Second, they point out that most of the criticisms apply to a generic model, but they assert that the various models differ on different dimensions and thus should not be treated identically. Finally, they propose a new twoprocess theory that they believe is invulnerable to the common criticisms. In what follows, I briefly comment on each of these points. I close my commentary by raising two questions: First, I ask whether the current model, like its two-system predecessors, is testable in any scientific sense and probe whether the empirical evidence, brought by proponents of the theory, really supports their claims. Second, I question whether the ontological inquiry about the existence 483474 PPSXXX10.1177/1745691613483474KerenA Tale of Two Systems research-article2013

On between-subjects versus within-subjects comparisons in testing utility theory☆

Abstract Empirical studies of expected utility theory often employ a between-subjects design. This practice has been recently criticized by J. C. Hershey and P. J. H. Schoemaker (1980 , Organizational Behavior and Human Performance , 25 , 395–418). The present paper provides a critical analysis of the controversial issues concerning the use of between-subjects vs within-subjects comparisons. It is claimed that the choice of experimental design should be determined, among other things, by theoretical aspects (e.g., the interpretation of utility theory) and the nature of the scientific problem. Following, we present relevant psychological considerations and conclude that, in the context of testing utility theory, a between-subjects design will often be more desirable. We then describe three different hypotheses that a researcher may be interested in testing, and identify the appropriate design for testing each of these hypotheses. The relationships between the different hypotheses are discussed. We apply our framework to reanalyze the reflection effect and compare it with the analysis proposed by Hershey and Schoemaker (1980) . Methodological implications for future research are briefly discussed.

Calibration of probability assessments by professional blackjack dealers, statistical experts, and lay people.

Abstract The question addressed in this study is whether experts are better calibrated than lay people. We investigated how well people are calibrated when they assess the probabilities of card combinations in the game of blackjack. Three groups of subjects were used: professional dealers, statistical experts, and control subjects. The results showed that experience and statistical expertise do not make people better calibrated in this task. It is argued that the concept of calibration is not wholly appropriate to describe the obtained deviations from the normatively correct responses. This is illustrated by a discriminant analysis performed on the signed deviation scores, which resulted in an almost perfect separation of the three groups, although they were overlapping with respect to calibration.

When are successes more surprising than failures

Surprise has been described in various contexts as a neutral, a positive, or a negative emotion. Six experiments are reported in which surprise ratings of unexpected positive and negative outcomes, with identical prior probabilities, were compared. For unexpected occurrences of events that are beyond the control of the protagonist, successes were consistently perceived as more surprising than failures. For unexpected action controlled outcomes for which the protagonist was partly responsible, failures were rated as more surprising than successes. The findings are interpreted as being due to an implicit contrast between out-of-control situations and success, in the first case, and between goal-directed efforts and failure, in the second.

Resolving social conflicts through hostage posting: Theoretical and empirical considerations

The well-known Prisoners Dilemma is often used to model situations of social conflict between 2 or more actors. The essence of the dilemmas is that each actor has individual incentives to act according to narrow self-interests referred to as defection, yet all actors are better off and receive higher payoffs for mutual cooperation. Many attempts have been made to study ways by which cooperation can be enhanced. This article focuses on a specific device, namely hostage posting or pledging of a bond. In the 1st part, a game-theoretic model is sketched that specifies the conditions under which hostage posting will effectively promote cooperative behavior. In the 2nd part, the authors describe several experiments that were designed to test the descriptive power of the theory, and at the same time they provide further insight into the processes underlying social conflicts. Implications regarding social conflicts and possible applications to other domains are briefly discussed in the end.

Temporal aspects of probabilistic predictions

Subjects who considered themselves to be experts made probability assessments regarding the outcomes of 108 soccer games. The mean number of correct predictions was less than that obtained by a linear model using type of game (home vs. outside games) and teams’ rankings as predictors. From a normative viewpoint, other things being equal, the more remote (in the future) an event, the less certain one should be regarding the outcome. Subjects apparently fail to take into account this temporal variable.