Ulrich Hoffrage

Probabilistic mental models : a brunswikian theory of confidence

Research on peoples confidence in their general knowledge has to date produced two fairly stable effects, many inconsistent results, and no comprehensive theory. We propose such a comprehensive framework, the theory of probabilistic mental models (PMM theory). The theory (a) explains both the overconfidence effect (mean confidence is higher than percentage of answers correct) and the hard-easy effect (overconfidence increases with item difficulty) reported in the literature and (b) predicts conditions under which both effects appear, disappear, or invert. In addition, (c) it predicts a new phenomenon, the confidence-frequency effect, a systematic difference between a judgment of confidence in a single event (i.e., that any given answer is correct) and a judgment of the frequency of correct answers in the long run. Two experiments are reported that support PMM theory by confirming these predictions, and several apparent anomalies reported in the literature are explained and integrated into the present framework.

Using natural frequencies to improve diagnostic inferences

PURPOSE: To test whether physicians diagnostic inferences can be improved by communicating information using natural frequencies instead of probabilities. Whereas probabilities and relative frequencies are normalized with respect to disease base rates, natural frequencies are not normalized. METHOD: The authors asked 48 physicians in Munich and DÃsseldorf to determine the positive predictive values (PPVs) of four diagnostic tests. Information presented in the four problems appeared either as probabilities (the traditional way) or as natural frequencies. RESULTS: When the information was presented as probabilities, the physicians correctly estimated the PPVs in only 10% of cases. When the same information was presented as natural frequencies, that percentage increased to 46%. CONCLUSION: Representing information in natural frequencies is a fast and effective way of facilitating diagnosis insight, which in turn helps physicians to better communicate risks to patients, and patients to better understand these risks.

The role of representative design in an ecological approach to cognition.

Egon Brunswik argued that psychological processes are adapted to environmental properties. He proposed the method of representative design to capture these processes and advocated that psychology be a science of organism-environment relations. Representative design involves randomly sampling stimuli from the environment or creating stimuli in which environmental properties are preserved. This departs from systematic design. The authors review the development of representative design, examine its use in judgment and decision-making research, and demonstrate the effect of design on research findings. They suggest that some of the practical difficulties associated with representative design may be overcome with modern technologies. The importance of representative design in psychology and the implications of this method for ecological approaches to cognition are discussed.

Fast, frugal, and fit: Simple heuristics for paired comparison

This article provides an overview of recent results on lexicographic, linear, and Bayesian models for paired comparison from a cognitive psychology perspective. Within each class, we distinguish subclasses according to the computational complexity required for parameter setting. We identify the optimal model in each class, where optimality is defined with respect to performance when fitting known data. Although not optimal when fitting data, simple models can be astonishingly accurate when generalizing to new data. A simple heuristic belonging to the class of lexicographic models is Take The Best (Gigerenzer & Goldstein (1996) Psychol. Rev. 102: 684). It is more robust than other lexicographic strategies which use complex procedures to establish a cue hierarchy. In fact, it is robust due to its simplicity, not despite it. Similarly, Take The Best looks up only a fraction of the information that linear and Bayesian models require; yet it achieves performance comparable to that of models which integrate information. Due to its simplicity, frugality, and accuracy, Take The Best is a plausible candidate for a psychological model in the tradition of bounded rationality. We review empirical evidence showing the descriptive validity of fast and frugal heuristics.

AIDS counselling for low-risk clients.

This study addresses the counselling of heterosexual men with low-risk behaviour who, voluntarily or involuntarily, take an HIV test. If such a man tests positive, the chance that he is infected can be as low as 50%. We study what information counsellors communicate to clients concerning the meaning of a positive test and whether they communicate this information in a way the client can understand. To get realistic data, one of us visited as a client 20 public health centres in Germany to take 20 counselling sessions and HIV tests. A majority of the counsellors explained that false positives do not occur, and half of the counsellors told the client that if he tests positive, it is 100% certain that he is infected with the virus. Counsellors communicated numerical information in terms of probabilities rather than absolute frequencies, became confused and were inconsistent. Based on experimental evidence, we propose a simple method that counsellors can learn to communicate risks in a more effective way.

The Reiteration Effect in Hindsight Bias

Repetition of an assertion increases the degree of belief in that assertion. This reiteration effect is used to explain two puzzling findings in research on hindsight bias. First, the reiteration effect explains the asymmetry in hindsight bias for true and false assertions. This striking asymmetry has often been observed in experimental studies, but no rationale has yet been found. Second, the reiteration effect predicts a novel finding: Recalled confidence will increase in hindsight bias studies even if no feedback is given. The authors have checked both predictions against results reported in the literature; with some exceptions, the evidence supports them.

Fast and Frugal Heuristics Are Plausible Models of Cognition: Reply to Dougherty, Franco-Watkins, and Thomas (2008).

M. R. Dougherty, A. M. Franco-Watkins, and R. Thomas (2008) conjectured that fast and frugal heuristics need an automatic frequency counter for ordering cues. In fact, only a few heuristics order cues, and these orderings can arise from evolutionary, social, or individual learning, none of which requires automatic frequency counting. The idea that cue validities cannot be computed because memory does not encode missing information is misinformed; it implies that measures of co-occurrence are incomputable and would invalidate most theories of cue learning. They also questioned the recognition heuristics psychological plausibility on the basis of their belief that it has not been implemented in a memory model, although it actually has been implemented in ACT-R (L. J. Schooler & R. Hertwig, 2005). On the positive side, M. R. Dougherty et al. discovered a new mechanism for a less-is-more effect. The authors of the present article specify minimal criteria for psychological plausibility, describe some genuine challenges in the study of heuristics, and conclude that fast and frugal heuristics are psychologically plausible: They use limited search and are tractable and robust.

Visual representation of statistical information improves diagnostic inferences in doctors and their patients

Doctors and patients have difficulty inferring the predictive value of a medical test from information about the prevalence of a disease and the sensitivity and false-positive rate of the test. Previous research has established that communicating such information in a format the human mind is adapted to-namely natural frequencies-as compared to probabilities, boosts accuracy of diagnostic inferences. In a study, we investigated to what extent these inferences can be improved-beyond the effect of natural frequencies-by providing visual aids. Participants were 81 doctors and 81 patients who made diagnostic inferences about three medical tests on the basis of information about prevalence of a disease, and the sensitivity and false-positive rate of the tests. Half of the participants received the information in natural frequencies, while the other half received the information in probabilities. Half of the participants only received numerical information, while the other half additionally received a visual aid representing the numerical information. In addition, participants completed a numeracy scale. Our study showed three important findings: (1) doctors and patients made more accurate inferences when information was communicated in natural frequencies as compared to probabilities; (2) visual aids boosted accuracy even when the information was provided in natural frequencies; and (3) doctors were more accurate in their diagnostic inferences than patients, though differences in accuracy disappeared when differences in numerical skills were controlled for. Our findings have important implications for medical practice as they suggest suitable ways to communicate quantitative medical data.

How to Keep Children Safe in Traffic: Find the Daredevils Early

Crossing the street in front of oncoming vehicles poses serious danger to young children. But is each young pedestrian similarly at risk? The authors aimed to identify children who are particularly prone to making risky and potentially harmful crossing decisions. They used a simple game involving risk to classify 5- to 6-year-olds as risk takers or risk avoiders. Children classified as risk takers made more crossing decisions at a busy 1-way street than risk avoiders, tolerated shorter time intervals between initiation of the crossing decision and arrival of the next vehicle, and were more likely to cause a (hypothetical) accident. Finally, they made decisions more quickly than risk avoiders. The authors discuss the implication of these results for traffic safety programs.

When one cue is not enough : Combining fast and frugal heuristics with compound cue processing

One-reason decision-making heuristics as proposed by Gigerenzer, Todd, and the ABC Research Group (1999) have been shown to perform accurately. However, such strategies cannot deal with compound cues. We propose the Take The Best Configural Cue (TTB-Configural) as a fast and frugal heuristic that processes compound cues. In a series of three experiments, we analysed whether participants used this heuristic when making cue-based inferences on which of two alternatives had a higher criterion value. In two of the experiments, two cues were amalgamated into a valid compound cue by applying the AND or the OR logical rule, respectively. In the third experiment, there was no valid compound cue. Within each experiment, we also manipulated causal mental models through instructions. In the configural causal model, cues were said to act through the same causal mechanism. In the elemental causal model, cues were said to act through different causal mechanisms. In the neutral causal model, the causal mechanism was not specified. When a highly valid compound existed, and participants had a configural causal model, for the majority of them the strategy that could best account for their choices was TTB-Configural. Otherwise, the strategy that best predicted their choices was the Take The Best (TTB) heuristic.