Jonathan J. Koehler

Betrayal Aversion: When Agents of Protection Become Agents of Harm

A form of betrayal occurs when agents of protection cause the very harm that they are entrusted to guard against. Examples include the military leader who commits treason and the exploding automobile air bag. We conducted five studies that examined how people respond to criminal betrayals, safety product betrayals, and the risk of future betrayal by safety products. We found that people reacted more strongly (in terms of punishment assigned and negative emotions felt) to acts of betrayal than to identical bad acts that do not violate a duty or promise to protect. We also found that, when faced with a choice among pairs of safety devices (air bags, smoke alarms, and vaccines), most people preferred inferior options (in terms of risk exposure) to options that included a slim (0.01%) risk of betrayal. However, when the betrayal risk was replaced by an equivalent non-betrayal risk, the choice pattern was reversed. Apparently, people are willing to incur greater risks of the very harm they seek protection from to avoid the mere possibility of betrayal.

Can Jurors Understand Probabilistic Evidence

Some courts have been reluctant to admit testimony expressing probabilities because of a concern that jurors will overweight it relative to other evidence. However, empirical studies indicate a tendency to underweight statistical evidence when other sources of evidence are available. For more than two decades, researchers have studied the ways that people process probabilistic and statistical information, but only a small portion of these studies focuses on the capacity of jurors to process explicitly quantitative probabilistic evidence. This paper reviews this research. It concludes that the work has produced several insights into the factors that affect the judgments of mock jurors, and that it is valuable in devising optimal rules for the admission or exclusion of probability evidence. At the same time, we do not believe that the experiments published to date have been adequate in their design and implementation to demonstrate unequivocally the extent to which jurors attend to trace evidence or to identify what decision aids, if any, would promote an appropriate weighting of the evidence at trial.

When are People Persuaded by DNA Match Statistics

The way in which statistical DNA evidence is presented to legal decision makers can have a profound impact on the persuasiveness of that evidence. Evidence that is presented one way may convince most people that the suspect is almost certainly the source of DNA evidence recovered from a crime scene. However, when the evidence is presented another way, a sizable minority of people equally convinced that the suspect is almost certainly not the source of the evidence. Three experiments are presented within the context of a theory (“exemplar cueing theory”) for when people will find statistical match evidence to be more and less persuasive. The theory holds that the perceived probative value of statistical match evidence depends on the cognitive availability of coincidental match exemplars. When legal decision makers find it hard to imagine others who might match by chance, the evidence will seem compelling. When match exemplars are readily available, the evidence will seem less compelling. Experiments 1 and 2 show that DNA match statistics that target the individual suspect and that are framed as probabilities (i.e., “The probability that the suspect would match the blood drops if he were not their source is 0.1%”) are more persuasive than mathematically equivalent presentations that target a broader reference group and that are framed as frequencies (“One in 1,000 people in Houston would also match the blood drops”). Experiment 3 shows that the observed effects are less likely to occur at extremely small incidence rates. Implications for the strategic use of presentation effects at trial are considered.

Thinking About Low-Probability Events: An Exemplar Cuing Theory

The way people respond to the chance that an unlikely event will occur depends on how the event is described. We propose that people attach more weight to unlikely events when they can easily generate or imagine examples in which the event has occurred or will occur than when they cannot. We tested this idea in two experiments with mock jurors using written murder scenarios. The results suggested that jurors attach more weight to the defendants claim that an incriminating DNA match is merely coincidental when it is easy for them to imagine other individuals whose DNA would also match than when it is not easy for them to imagine such individuals. We manipulated the difficulty of imagining such examples by varying the description of the DNA-match statistic. Some of the variations that influenced the jurors were normatively irrelevant.

Selection Neglect in Mutual Fund Advertisements

Mutual fund companies selectively advertise their better-performing funds. However, investors respond to advertised performance data as if those data were unselected (i.e., representative of the population). We identify the failure to discount selected or potentially selected data as selection neglect. We examine these phenomena in an archival study (Study 1) and two controlled experiments (Studies 2 and 3). Study 1 identifies selection bias in mutual fund advertising by showing that the median performance rank for advertised funds is between the 79th and 100th percentile. Study 2 finds that both novice investors and financial professionals fall victim to selection neglect in a financial advertising task unless the advertisement makes the selective nature of available performance data transparent. Study 3 shows that selection neglect associated with a large well-known company can be debiased with a simple extrinsic sample space cue, although individual differences in statistical reasoning also matter. We argue that selection neglect results from a general tendency to ignore underlying sample spaces rather than a fundamental misunderstanding about the data selection process or the value of selected data.

The Misquantification of Probative Value

D. Davis and W. C. Follette (2002) purport to show that when “the base rate” for a crime is low, the probative value of “characteristics known to be strongly associated with the crime ... will be virtually nil.” Their analysis rests on the choice of an arbitrary and inapposite measure of the probative value of evidence. When a more suitable metric is used (e.g., a likelihood ratio), it becomes clear that evidence they would dismiss as devoid of probative value is relevant and diagnostic.

Probabilities in the Courtroom: An Evaluation of the Objections and Policies

This quotation is notable both for its wisdom and its issuance from a dissenting opinion. We live in a probabilistic world. There are few guarantees, and only rarely does the search for truth end with a certain answer. More often, the truths we discover are generalizations, statements that predict outcomes in an implicitly probabilistic fashion.

Forensic bitemark identification: weak foundations, exaggerated claims

Abstract Several forensic sciences, especially of the pattern-matching kind, are increasingly seen to lack the scientific foundation needed to justify continuing admission as trial evidence. Indeed, several have been abolished in the recent past. A likely next candidate for elimination is bitemark identification. A number of DNA exonerations have occurred in recent years for individuals convicted based on erroneous bitemark identifications. Intense scientific and legal scrutiny has resulted. An important National Academies review found little scientific support for the field. The Texas Forensic Science Commission recently recommended a moratorium on the admission of bitemark expert testimony. The California Supreme Court has a case before it that could start a national dismantling of forensic odontology. This article describes the (legal) basis for the rise of bitemark identification and the (scientific) basis for its impending fall. The article explains the general logic of forensic identification, the claims of bitemark identification, and reviews relevant empirical research on bitemark identification—highlighting both the lack of research and the lack of support provided by what research does exist. The rise and possible fall of bitemark identification evidence has broader implications—highlighting the weak scientific culture of forensic science and the laws difficulty in evaluating and responding to unreliable and unscientific evidence.

Betrayal aversion is reasonable

This article was written in response to Cass Sunstein’s, Moral Heuristics. We accept Sunstein’s claim that people often use moral heuristics to make judgments and decisions. Indeed, given people’s desire for social goals such as fairness, justice, and trustworthiness, it would be strange if moral intuitions did not impact the decisions people make. However, it is less clear that these moral intuitions - or moral heuristics - are as prone to systematic error as the classic heuristics in situations that include a risk of betrayal. We disagree with Sunstein about when the relevant moral heuristic may be said to “misfire”. We suggest that the moral heuristic people apply to avoid the possibility of safety-product betrayal may be reasonable.

Misconceptions About Statistics and Statistical Evidence

Thanks in large part to advances in computing and information technology, statistics are everywhere. Whether the information concerns business, health, politics, sports, or nearly anything else, it is likely to appear in statistical form. The front page of the country’s highest circulation daily newspaper (USA Today) is littered with descriptive statistics and graphical depictions of those statistics. The star of a popular prime-time television show called Numbers solves fictional legal cases each week through the innovative use of statistics and statistical reasoning. And so, it is no surprise that statistics and statistical arguments find their way into the American courtroom at an unprecedented rate. Fienberg (1989 reported “dramatic growth” in the use of statistical evidence from the 1960s through the 1980s. He noted that the terms “statistic” or “statistical” appeared in thousands of reported district court opinions (p. 7). I performed a Westlaw search on these terms and found a 56% increase in the use of these terms in the Federal Cases database from 1990 to 2004. I also found that the phrase “statistical analysis” appeared 94% more often in 2004 than in 1990; “regression analysis” appeared 95% more often.