James Shanteau

Competence in experts: The role of task characteristics

Abstract The previous literature on experts presents two contrasting views. Judgment and decision research has shown that experts make flawed decisions due, in part, to the biasing effects of judgmental heuristics. Cognitive science research, in contrast, views experts as competent and different from novices in nearly every aspect of cognitive functioning. An alternative view developed here, the Theory of Expert Competence, suggests that both analyses are correct, but incomplete. In particular, the theory assumes competence depends on five components: (1) a sufficient knowledge of the domain, (2) the psychological traits associated with experts, (3) the cognitive skills necessary to make tough decisions, (4) the ability to use appropriate decision strategies, and (5) a task with suitable characteristics. The latter is the focus of this paper. Insufficient attention has been paid to task and domain characteristics in prior research. Decision researchers have looked primarily at experts in behavioral domains (e.g., clinical psychology), whereas cognitive investigators have concentrated on experts in static domains (e.g., physics). Thus, the discrepancy in the conclusions drawn from the two literatures appears to be a function of the different domains studied. Although similar to approaches such as Cognitive Continuum Theory, the proposed theory contains several new components. In addition, the theory has implications both for the analysis of experts and for the design and use of expert systems.

Psychological characteristics and strategies of expert decision makers

Abstract Previous studies of expert decision makers have concluded that experts, because of cognitive limitations, are generally inaccurate, unreliable, biased, lack self-insight, and gain little with experience. Overall, previous psychological studies have painted a rather bleak picture of the decision-making abilities of experts. The research reviewed here provides a different view of experts in two respects. First, expert decision makers have been found to use strategies, such as reliance on group feedback, willingness to make adjustments, and a divide-and-conquer approach, which help them overcome the effects of cognitive limitations. Second, top decision makers in agriculture, personnel selection, health care, accounting/auditing, and management have been observed to share psychological characteristics such as perceptiveness, communication skills, self-confidence, and creativity under stress. These findings have implications for (1) image and expectations of experts, (2) classifying different types of experts, (3) training and/or selecting novices to become experts, and (4) design of expert systems.

How much information does an expert use ? is it relevant ?

Abstract Many observers of expert decision makers have assumed an Information-Use Hypothesis: The amount of information used, as measured by number of significant cues, should be greater for experts than non-experts. Since prior studies consistently have shown that both expert and naive judgment can be described using few cues, the conclusion has been drawn that experts are limited decision makers. This paper takes a new look at this conclusion by reviewing recent literature on information use of experts and by presenting some new evidence. The results from five studies show that experts often have the same (or fewer) number of significant cues as novices, but that the information used is more relevant. Therefore, the amount of information used does not reflect degree of expertise; however, the type of information used does. This finding has implications for measurement of expertise, analysis of expert tasks, and generalizability of conclusions about experts.

Livestock judges: How much information can an expert use?☆☆☆

Abstract The number of dimensions used by expert livestock judges in making decisions was determined by two different experimental procedures. In the first, judgments were made of hypothetical gilts (female breeding pigs) described by verbal statements along 11 relevant dimensions. In the second, judgments were made based on photographs of Poland-China breeding gilts. The judges were found to use 9–11 pieces of information in the first study but were generally found to use fewer than three in the second. These results were interpreted to indicate that expert judges can integrate a large number of dimensions but that intercorrelations present in real stimuli tend to reduce the number of dimensions found. This suggests that experts may be able to use substantially more information than was previously thought.

Performance-based assessment of expertise: How to decide if someone is an expert or not

Abstract The identification of an expert is vital to any study or application involving expertise. If external criterion (a “gold standard”) exists, then identification is straightforward: Simply compare people against the standard and select whoever is closest. However, such criteria are seldom available for domains where experts work; thats why experts are needed in the first place. The purpose here is to explore various methods for identifying experts in the absence of a gold standard. One particularly promising approach (labeled CWS for Cochran–Weiss–Shanteau ) is explored in detail. We illustrate CWS through reanalyses of three previous studies of experts. In each case, CWS provided new insights into identifying experts. When applied to auditors, CWS correctly detected group differences in expertise. For agricultural judges, CWS revealed subtle distinctions between subspecialties of experts. In personnel selection, CWS showed that irrelevant attributes were more informative than relevant attributes. We believe CWS provides a valuable tool for identification and evaluation of experts.

Reducing the influence of irrelevant information on experienced decision makers

Abstract In this study two training procedures designed to reduce the adverse influence of irrelevant information were tested for their efficacy. One training procedure involved a lecture (paralleling typical classroom experience), while the other involved interaction and practice (paralleling typical laboratory experience). In a pretest, irrelevant information was shown to influence the judgments of 12 experienced student soil judges. The judges were then given lecture training; this was found to be of minimal help in reducing the influence of irrelevance. Next, the judges received the interactive training and that had a significant impact. In addition to reducing the influence of irrelevance, the interactive training also improved the accuracy of the judgments. In a followup study, five of the soil judges were reevaluated over a year later; the training appeared to have continued impact. Implications and extensions of the training procedures to other areas are discussed.

The Psychology of Experts An Alternative View

These quotes illustrate two facts: First, the topic of experts and expertise is of interest to many writers, both in literature and in science. Second, these writers disagree about the value of advice from experts. The goal of this paper is to explore the issue of expertise from a psychological perspective and to use that perspective to provide insights into the differing views of experts.

Empirical assessment of expertise.

The assessment of expertise is vital both in practical situations that call for expert judgment and in theoretical research on the psychology of experts. It can be difficult, however, to determine whether a judge is in fact performing expertly. Our goal was to develop an empirical measure of expert judgment. We argue that two necessary characteristics of expertise are discrimination of the various stimuli in the domain and consistent treatment of similar stimuli. We combine measures of these characteristics to form a ratio we call the Cochran-Weiss-Shanteau (CWS) index of expertise. The proposed index was demonstrated using two studies that distinguished experts from nonexperts based on their judgmental performance. The index provides new insights into expertise and offers a partial definition of expertise that may be useful in a variety of theoretical and applied settings. Potential applications of this research include selection, training, and evaluation of experts and of expert-machine systems.

Psychological characteristics of expert decision makers

“When the expert entered the room, everyone knew it. There was a kind of aura which surrounded him.”

An information processing view of framing effects: The role of causal schemas in decision making

People prefer a sure gain to a probable larger gain when the two choices are presented from a gain perspective, but a probable larger loss to a sure loss when the objectively identical choices are presented from a loss perspective. Such reversals of preference due to the context of the problem are known asframing effects. In the present study, schema activation and subjects’ interpretations of the problems were examined as sources of the framing effects. Results showed that such effects could be eliminated by introducing into a problem a causal schema that provided a rationale for the reciprocal relationship between the gains and the losses. Moreover, when subjects were freed from framing they were consistently risk seeking in decisions about human life, but risk averse in decisions about property. Irrationality in choice behaviors and the ecological implication of framing effects are discussed.