David Faust

Eliminating the hindsight bias.

Those who consider the likelihood of an event after it has occurred exaggerate their likelihood of having been able to predict that event in advance. We attempted to eliminate this hindsight bias among 194 neuropsychologists. Foresight subjects read a case history and were asked to estimate the probability of three different diagnoses. Subjects in each of the three hindsight groups were told that one of the three diagnoses was correct and were asked to state what probability they would have assigned to each diagnosis if they were making the original diagnosis. Foresight-reasons and hindsight-reasons subjects performed the same task as their foresight and hindsight counterparts, except they had to list one reason why each of the possible diagnoses might be correct. The frequency of subjects succumbing to the hindsight bias was lower in the hindsight-reasons groups than in the hindsight groups not asked to list reasons, x!( 1, N = 140) = 4.12, p < .05.

A national survey of psychologists who offer neuropsychological services.

We conducted a national survey of psychologists who offer neuropsychological services to determine levels of training, current practices, and views on professional issues. All subjects were listed in the National Register of Health Service Providers in Psychology and/or the American Psychological Association Directory as having some affiliation with neuropsychology. Results suggest marked diversity within the field, with the modal practitioner being minimally involved in neuropsychological activities. We also compare clinicians with varying involvement and experience in neuropsychology and analyze the concentration of practitioners by geographic regions. We compare our results to those obtained in prior surveys and discuss implications of the findings.

Practicing Psychologists' Knowledge of General Psychotherapy Research Findings: Implications for Science-Practice Relations

If you are a therapist, how knowledgeable are you and how knowledgeable do you need to be about psychotherapy research findings? In this study, the authors examined practicing psychologists’ knowledge of general psychotherapy research findings. Results revealed that some psychologists showed excellent familiarity with this body of outcome research, but many did not achieve this standard. Not infrequently, psychologists believed that research findings were less positive than is actually the case, perhaps explaining some of the negativity that practitioners sometimes express toward psychotherapy research. Research knowledge could not be predicted by years graduated, percentage of long-term clients, percentage of time conducting therapy, theoretical orientation, or perceived familiarity with research. The modest familiarity with research findings that therapists, in general, demonstrated may be understood, in part, through examination of the acquisition of research knowledge as a judgment task. The authors explore potential factors that may influence therapists’ judgments about the research. In addition, they examine possible relations between research knowledge and therapy outcome and their potential practice implications.

Leading researchers' consensus on psychotherapy research findings: Implications for the teaching and conduct of psychotherapy

We examined leading international psychotherapy researchers’ views on psychotherapy outcome research. Participants completed a questionnaire on which they rated level of research evidence for or against various assertions about psychotherapy processes and outcomes. Participants rated how confident they were that the assertions were supported by psychotherapy research. Strong, or relatively strong, consensus was achieved on several of the questionnaire items. Areas for which relative uniformity of opinion does or does not exist have potential implications for the teaching and conduct of psychotherapy and for the science–practice interface in psychotherapy. Additionally, consensus about psychotherapy findings can be used as a yardstick by which to measure practicing clinicians’ knowledge of the research. In 1949 at the Boulder Conference, the profession of clinical psychology declared its allegiance to the goal of integrating science and practice and cast the field in the “scientist–practitioner model,” or the Boulder Model (Raimy, 1950). Additionally, at the Vail Conference in 1973, the profession of clinical psychology addressed the science–practice interface but endorsed a “professional model” of training focused on practitioner training issues and psychologists as consumers and users of research (Korman, 1974). Both the Boulder and Vail Conferences emphasized the knowledge and use of psychological research in clinical practice. A central mission of the profession of psychology was to accumulate scientific knowledge in order to advance the understanding

Using Meta‐Scientific Studies to Clarify or Resolve Questions in the Philosophy and History of Science

More powerful methods for studying and integrating the historical track record of scientific episodes and scientific judgment, or what Faust and Meehl describe as a program of meta‐science and meta‐scientific studies, can supplement and extend more commonly used case study methods. We describe the basic premises of meta‐science, overview methodological considerations, and provide examples of meta‐scientific studies. Meta‐science can help to clarify or resolve long‐standing questions in the history and philosophy of science and provide practical help to the working scientist.

Social-Emotional Assessment Practices in School Psychology

This study examined the social-emotional assessment practices of a random sample of school psychologists in the United States. For each assessment instrument, respondents indicated frequency of use, rationale for use, information gained, importance, and scoring system used. Results indicate that projective tests remain popular, are used primarily to generate hypotheses about social-emotional functioning, are viewed as important in the assessment process, are applied for a range of purposes, and often are scored with personalized systems. Most objective tests are used to confirm hypotheses about social-emotional functioning, also are viewed as important in the assessment process, and are applied for various purposes. Surprisingly, a considerable percentage of school psychologists use personalized systems to score some objective tests. Implications of the findings are discussed.

Improving the Accuracy of Clinical Judgment

Diagnostic reasoning in psychiatry is as intricate as it is risky (Elstein et al. 1978; Fitzhenry-Coor 1986; Fitzhenry-Coor and Nurcombe 1983; Gauron and Dickinson 1965; Sandifer et al. 1970; Kendell 1973; Nurcombe and Fitzhenry-Coor 1982). During the clinical encounter, the clinician elicits, notes, and verifies pertinent clinical cues. Early in the encounter, from an assembled but incomplete pattern of data, an array of diagnostic hypotheses is generated. Subsequently, in the light of evidence systematically gathered, each diagnostic possibility is progressively refined or deleted until the diagnostic conclusion is reached. The validity of the clinical endeavor, thus, depends upon the pertinence of the hypotheses generated, for it is upon them that deductive reasoning will hinge. It also depends upon the reliability of the history, signs, and investigations that constitute the evidence gathered for or against each of the hypotheses. The purpose of this paper is to propose systematic strategies based on principles of probability and empirical research, with the aim of sharpening clinical reasoning.

Forensic neuropsychology: The art of practicing a science that does not yet exist

Despite its future promise, neuropsychological evidence generally lacks scientifically demonstrated value for resolving legal issues, and thus, if admitted into court, should be accorded little or no weight. In support of this contention, examples of problems and limits in forensic neuropsychology are described. These include contrasts between the clinical and forensic context; the base-rate problem; lack of standardized practices; problems assessing credibility or malingering; difficulties determining prior functioning, limits in the capacity to integrate complex data; and the lack of relation between judgmental accuracy and education, experience, or credentials. Some possible counterarguments are also addressed.

Errors in Scoring Objective Personality Tests

Given the paucity of previous research, we examined the occurrence of scoring error on widely used objective personality tests and examined its possible relation to two factors: scoring procedure complexity (SPC) and commitment to accuracy (CTA). We doublechecked the scoring of three tests (MMPI, Beck Depression Inventory, Spielberger State/Trait Anxiety Inventory) across three settings. Each of the tests were misscored at a surprisingly high rate in at least one setting, and some such errors altered major interpretive implications. Tests of higher SPC showed greater error rates, but high CTA greatly reduced the occurrence of error across levels of SPC. Unexpected sources of error were also uncovered, such as commercial computer scoring errors and disagreement in scoring standards among test publishers. Practical suggestions for improving scoring accuracy are offered.

School psychologists' diagnoses of learning disabilities: A study of illusory correlation.

Abstract We examined whether school psychologists based diagnoses of learning disabilities on irrelevant data. Nationally certified school psychologists received case study material containing both relevant and irrelevant information for diagnosing a learning disability. The psychologists indicated (a) the probability that the client was learning-disabled, (b) their confidence in the diagnosis, and (c) their decision-making strategy. They tended to base diagnostic decisions on false beliefs, which we discuss in respect to prior research on illusory correlation.