Paul M. Spengler

The Meta-Analysis of Clinical Judgment Project: Fifty-Six Years of Accumulated Research on Clinical Versus Statistical Prediction:

Clinical predictions made by mental health practitioners are compared with those using statistical approaches. Sixty-seven studies were identified from a comprehensive search of 56 years of research; 92 effect sizes were derived from these studies. The overall effect of clinical versus statistical prediction showed a somewhat greater accuracy for statistical methods. The most stringent sample of studies, from which 48 effect sizes were extracted, indicated a 13% increase in accuracy using statistical versus clinical methods. Several variables influenced this overall effect. Clinical and statistical prediction accuracy varied by type of prediction, the setting in which predictor data were gathered, the type of statistical formula used, and the amount of information available to the clinicians and the formulas. Recommendations are provided about when and under what conditions counseling psychologists might use statistical formulas as well as when they can rely on clinical methods. Implications for clinical judgment research and training are discussed.

Clinical Judgmental Biases: The Moderating Roles of Counselor Cognitive Complexity and Counselor Client Preferences.

This study found that counselor individual differences in cognitive complexity, but not preferences for client problems, moderate the cognitive processes that lead to bias in clinical judgment. A particularly robust and unambiguous clinical bias, known as diagnostic overshadowing, was selected for the study.

Should I Pack My Umbrella? Clinical Versus Statistical Prediction of Mental Health Decisions

In this rejoinder, the authors respond to the insightful commentary of Strohmer and Arm, Chwalisz, and Hilton, Harris, and Rice about the meta-analysis on statistical versus clinical prediction techniques for mental health judgments. The authors address issues including the availability of statistical prediction techniques for real-life psychology applications, the development of these prediction techniques for future applications, and the training of counseling and other psychologists in using statistical prediction techniques. Many of these issues are couched in the historical debate about clinical versus statistical prediction.

A meta-analysis of confidence and judgment accuracy in clinical decision making.

The overconfidence bias occurs when clinicians overestimate the accuracy of their clinical judgments. This bias is thought to be robust leading to an almost universal recommendation by clinical judgment scholars for clinicians to temper their confidence in clinical decision making. An extension of the Meta-Analysis of Clinical Judgment (Spengler et al., 2009) project, the authors synthesized over 40 years of research from 36 studies, from 1970 to 2011, in which the confidence ratings of 1,485 clinicians were assessed in relation to the accuracy of their judgments about mental health (e.g., diagnostic decision making, violence risk assessment, prediction of treatment failure) or psychological issues (e.g., personality assessment). Using a random effects model a small but statistically significant effect (r = .15; CI = .06, .24) was found showing that confidence is better calibrated with accuracy than previously assumed. Approximately 50% of the total variance between studies was due to heterogeneity and not to chance. Mixed effects and meta-regression moderator analyses revealed that confidence is calibrated with accuracy least when there are repeated judgments, and more when there are higher base rate problems, when decisions are made with written materials, and for earlier published studies. Sensitivity analyses indicate a bias toward publishing smaller sample studies with smaller or negative confidence-accuracy effects. Implications for clinical judgment research and for counseling psychology training and practice are discussed.

Clinical Judgment of Major Depression in AIDS Patients: The Effects of Clinician Complexity and Stereotyping

This clinical judgment study found an overshadowing bias that existed in the treatment of major depression in AIDS patients. Two clinician individual differences—cognitive complexity about AIDS issues and attitudes toward AIDS victims—were investigated for possible moderating effects of the treatment overshadowing bias. Cognitive complexity about AIDS issues had a significant moderating effect, as more complex clinical and counseling psychologists were more likely to recommend antidepressant medication. Attitudes toward AIDS victims, measured by the Attitudes Towards AIDS Victims scale (Larsen, Serra, & Long, 1990), did not moderate clinicians treatment judgments. Results from a second set of exploratory analyses suggest that the diagnostic overshadowing occurred, but as a function of the presence of a terminal illness (AIDS or cancer) and not as a bias unique to AIDS issues. Implications for research and practice are discussed.

Helping and change without traditional therapy: Commonalities and opportunities

The therapeutic elements of four types of helping interactions (friends/family, religion, cinematherapy/bibliotherapy, and self-help groups), and their commonalities with traditional psychotherapy are explored. Empirical findings in these areas are discussed, along with suggestions for process and outcome research. Recommendations for the integration of self-help and paraprofessional counseling with professional therapy are presented.

A comprehensive meta-reanalysis of the robustness of the experience-accuracy effect in clinical judgment.

Experience is one of the most commonly studied variables in clinical judgment research. In a meta-analysis of research from 1970 to 1996 of judgments made by 4,607 participants from 74 studies, Spengler, White, Ægisdóttir, Maugherman, Anderson, et al. (2009) found an experience-accuracy fixed effect of d = .121 (95% CI [.06, .18]), indicating that with more experience, counseling and other psychologists obtain only modest gains in decision-making accuracy. We sought to conduct a more rigorous assessment of the experience-accuracy effect by synthesizing 40 years of research from 1970 to 2010, assessing the same and additional moderators, including subgroup analyses of extremes of experience, and conducting a sensitivity analysis. The judgments formed by 11,584 clinicians from 113 studies resulted in a random effects d of .146 (95% CI [.08, .21]), reflecting the robustness of only a small impact of experience on decision-making accuracy. The sensitivity analysis revealed that the effect is consistent across analysis and methodological considerations. Mixed effects metaregression revealed no statistically significant relation between 40 years of time and the experience-accuracy effect. A cumulative meta-analysis indicated that the experience-accuracy effect stabilized in the literature in the year 1999, after the accumulation of 82 studies, with no appreciable change since. We assessed a broader range of experience comparing no experience to some experience and comparing nonexperts with experts, and for differences as a function of decision making based on psychological tests; however, these and most other moderators were not significant. Implications are discussed for clinical decision-making research, training, and practice.

Stability of a 4 × 6 repertory grid for measuring cognitive complexity

Abstract Stability indices were determined for a 4 × 6 version of the 10 × 10 repertory grid of Bieri et al. (1966). Adequate correlation with complexity scores derived from the 10 × 10 grid, cross-validation between the two grids, and test-retest reliability were demonstrated. The 4 × 6 repertory grid provides a useful version of this measure of cognitive complexity for researchers who desire a valid, but smaller and less time-consuming, measure with provided constructs and role types.

Microaggressions: Clinical errors with sexual minority clients.

Sexual minority (SM) individuals live in a heterosexist society that denigrates their sexual orientation identity. The stigma and prejudice they regularly encounter is hypothesized to lead to their significantly increased risk for developing mental health disorders. Because of these factors, therapists should be diligent to create an affirming and supportive therapeutic environment but this is often not the case. SM clients frequently report experiencing sexual orientation microaggressions in therapy, such as heteronormative statements, a disregard for their sexual orientation identity, and an assumption that their presenting issues are rooted in their sexual orientation identity. These microaggressions should be viewed as bias manifested as clinical errors because of how they weaken therapeutic alliance, decrease the effectiveness of treatment, decrease utilization intent, and cultivate feelings of shame, anger, and misunderstanding. This article provides empirically supported findings regarding common SM clinical errors and microaggressions, a clinical example of such biases with corrective examples along with the authors personal reactions, and more general strategies for avoiding microaggressive errors with SM clients. Implications for practice, training, and research are discussed. (PsycINFO Database Record

Time Keeps on Ticking The Experience of Clinical Judgment

In this rejoinder, the authors respond to reactions by Ridley and Shaw-Ridley and by Lichtenberg to their meta-analysis on the effects of experience on clinical judgment decision making accuracy.