David R. Kraus

Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions.

Abstract This article reviews the benefits, obstacles, and challenges that can hinder (and have hindered) implementation of routine outcome monitoring in clinical practice. Recommendations for future routine outcome assessment efforts are also provided. Spanning three generations, as well as multiple developed tools and approaches, the four authors of this article have spent much of their careers working to address these issues and attempt to consolidate this learning and experience briefly here. Potential “elephants in the room” are brought into the discussion wherever relevant, rather than leaving them to obstruct silently the fields efforts. Some of these topics have been largely ignored, yet must be addressed if we are to fulfill our promise of integrating science and practice. This article is an attempt to identify these important issues and start an honest and open dialogue.

Therapist effectiveness: implications for accountability and patient care.

Abstract Significant therapist variability has been demonstrated in both psychotherapy outcomes and process (e.g., the working alliance). In an attempt to provide prevalence estimates of “effective” and “harmful” therapists, the outcomes of 6960 patients seen by 696 therapists in the context of naturalistic treatment were analyzed across multiple symptom and functioning domains. Therapists were defined based on whether their average client reliably improved, worsened, or neither improved nor worsened. Results varied by domain with the widespread pervasiveness of unclassifiable/ineffective and harmful therapists ranging from 33 to 65%. Harmful therapists demonstrated large, negative treatment effect sizes (d=−0.91 to −1.49) while effective therapists demonstrated large, positive treatment effect sizes (d=1.00 to 1.52). Therapist domain-specific effectiveness correlated poorly across domains, suggesting that therapist competencies may be domain or disorder specific, rather than reflecting a core attribute or underlying therapeutic skill construct. Public policy and clinical implications of these findings are discussed, including the importance of integrating benchmarked outcome measurement into both routine care and training.

The Treatment Outcome Package: Facilitating Practice and Clinically Relevant Research

As an effort to foster the use of standardized data in clinical practice, this article discusses an outcome measure developed by clinicians for use in naturalistic settings: the treatment outcome package (TOP). In addition to describing how the TOP can be used in day-to-day practice, the article illustrates how its multidimensional structure can inform several aspects of clinical work, such as the development of case formulations, facilitation of client-therapist communication, tracking (positive and negative) change during treatment, and the documentation of specific area of therapist expertise. This article recognizes the challenges and drawbacks that are associated with the use of a standardized outcome measure and also describes three ways by which the TOP can contribute to the strengthening of the relationship between research and practice.

Attachment history as a moderator of the alliance outcome relationship in adolescents.

The role of the alliance in predicting treatment outcome is robust and long established. However, much less attention has been paid to mechanisms of change, including moderators, particularly for youth. This study examined the moderating role of pretreatment adolescent-caregiver attachment and its impact on the working alliance-treatment outcome relationship. One hundred adolescents and young adults with primary substance dependence disorders were treated at a residential facility, with a cognitive-behavioral emphasis. The working alliance and clinical symptoms were measured at regular intervals throughout treatment. A moderator hypothesis was tested using a path analytic approach. Findings suggested that attachment to the primary caregiver moderated the impact of the working alliance on treatment outcome, such that for youth with the poorest attachment history, working alliance had a stronger relationship with outcome. Conversely, for those with the strongest attachment histories, alliance was not a significant predictor of symptom reduction. This finding may help elucidate alliance-related mechanisms of change, lending support for theories of corrective emotional experience as one function of the working alliance in youth psychotherapy.

Factor structure of the treatment outcome package for children

The Treatment Outcome Package (TOP) is a behavioral health assessment and outcome battery with modules that assess a wide array of behavioral health symptoms and functioning, demographics, case-mix variables, and treatment satisfaction. The TOP was developed to follow the design specifications set forth by the Core Battery Conference (Horowitz, Lambert, & Strupp, 1997), but also to ensure the batterys applicability to naturalistic treatment. The TOP also includes a child version that addresses recommendations recently voiced by several experts in youth assessment and treatment. The purpose of this manuscript is to document the process involved in developing the shortened version of the Child TOP. With large samples of diverse patients, 103 items were reduced to 48, with 13 stable and clinically useful subscales. The final confirmatory factor analysis of 7,267 patients replicated the model with excellent results. Although some of these subscales were similar to factors derived from the adult version of the TOP, others were specific to childhood disorders. In addition, the analyses demonstrated that the TOP is not restricted to manifestations of distress and impairment, but also captures a unique factor of childhood strengths (i.e., resiliency).

Validating the rapid responder construct within a practice research network.

OBJECTIVE The present study was a replication and extension of prior work (Stulz, Lutz, Leach, Lucock, & Barkham, ) that identified multiple groups of clients in treatment with high-symptom severity and markedly different recovery trajectories (rapid/early response vs. little or no response). METHOD Using data collected through repeated administrations of the Depression subscale of the Treatment Outcome Package (n = 147), growth mixture modeling was employed to determine whether clients fell into discrete groups of response trajectories during 15 sessions of psychotherapy. Additionally, logistic regressions were conducted to assess possible predictors of group membership. RESULTS Three separate groups of treatment responders were identified: 2 high-symptom groups-rapid responders and nonresponders-and 1 low-symptom group of nonresponders. Elevated social conflict and suicidality predicted increased likelihood of membership in the high-symptom nonresponder group. Increased feelings of interpersonal hostility and better sexual functioning predicted increased likelihood of membership in the rapid responder group. CONCLUSION Replication of earlier results provides further evidence for the usefulness of modeling change during psychotherapy using multiple trajectories. Predictors of group membership indicate the influence of functional impairment on recovery, and support the importance of multidimensional measurement of client problems.

The Expanding Relevance of Routinely Collected Outcome Data for Mental Health Care Decision Making

Abstract Evidence shows that routine outcome monitoring (ROM) and feedback using standardized measurement tools enhances the outcomes of individual patients. When outcome data from a large number of patients and clinicians are collected, patterns can be tracked and comparisons can be made at multiple levels. Variability in skills and outcomes among clinicians and service settings has been documented, and the relevance of ROM for decision making is rapidly expanding alongside the transforming health care landscape. In this article, we highlight several developing core implications of ROM for mental health care, and frame points of future work and discussion.

Treatment Outcome Package: Measuring and facilitating multidimensional change.

The Treatment Outcome Package (TOP; D. R. Kraus, Seligman, & Jordan, 2005) is a multidimensional routine progress and outcome measure developed for use in diverse naturalistic practice settings. In this article, we (a) provide a brief review and summary of the extant psychometric and research support for the TOP, (b) provide examples of the TOPs use in clinical training and practice, and (c) discuss the implications of the TOP for future psychotherapy training, research, and practice. In particular, we focus on the implications of risk-adjusted progress monitoring for systems of care and mental health care decision making.

Conducting research and collaborating with researchers: The experience of clinicians in a residential treatment center

Abstract This paper describes the experience of clinicians in conducting research and collaborating with academic researchers. As part of clinical routine of a residential program for adolescent substance abusers, empirical data have been collected to assess clients needs before and after treatment, improve clinical practice, and identify barriers to change. Some of the challenges faced and the benefits learned in conducting these studies are presented. In addition to highlighting the convergence of research interests between clinicians and academicians, the conclusion offers general recommendations to foster these partnerships and solidify the scientific-practitioner model.

Harnessing the Therapist Effect in Patient-Centered Mental Health Care Decision Making

In this commentary, we address the critical importance of the research literature documenting therapist differences in skill and outcomes. In particular, we focus on the implications of therapist differences for patient-centered decision making, including the matching of patients to specific psychotherapists who have empirically-based track records of positive performance. In addition, we present preliminary results from a grant supported study of patients’ values and preferences regarding the use of provider performance track records in routine mental health care decision making.