Chris Leach

Service profiling and outcomes benchmarking using the CORE-OM: Toward practice-based evidence in the psychological therapies.

To complement the evidence-based practice paradigm, the authors argued for a core outcome measure to provide practice-based evidence for the psychological therapies. Utility requires instruments that are acceptable scientifically, as well as to service users, and a coordinated implementation of the measure at a national level. The development of the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) is summarized. Data are presented across 39 secondary-care services (n = 2,710) and within an intensively evaluated single service (n = 1,455). Results suggest that the CORE-OM is a valid and reliable measure for multiple settings and is acceptable to users and clinicians as well as policy makers. Baseline data levels of patient presenting problem severity, including risk, are reported in addition to outcome benchmarks that use the concept of reliable and clinically significant change. Basic quality improvement in outcomes for a single service is considered.

Early sudden gains in psychotherapy under Routine clinic conditions: Practice-based evidence

Sudden gains--large, enduring reductions in symptom intensity from one session to the next--were identified by T. Z. Tang and R. J. DeRubeis (1999b) on the basis of data from 2 manualized clinical trials of cognitive therapy for depression. The authors found similar sudden gains among clients with a variety of disorders treated with a variety of approaches in routine clinic settings. Clients (N = 135 who met inclusion criteria) completed short forms of the Clinical Outcomes in Routine Evaluation (CORE-SF) preceding 7 to 74 individual sessions. Those who experienced sudden gains within their first 16 sessions (n = 23) had significantly lower CORE-SF scores in their final 3 sessions than did the other clients.

Predicting Change for Individual Psychotherapy Clients on the Basis of their Nearest Neighbors.

This study extended client-focused research by using the nearest neighbor (NN) approach, a client-specific sampling and prediction strategy derived from research on alpine avalanches. Psychotherapy clients (N=203) seen in routine practice settings in the United Kingdom completed a battery of intake measures and then completed symptom intensity ratings before each session. Forecasts of each clients rate of change and session-by-session variability were computed on the basis of that clients NNs (n=10-50 in different comparisons). Alternative forecasts used linear or log-linear slopes and were compared with an alternative prediction strategy. Results showed that the NN approach was superior to the alternative model in predicting rate of change, though the advantage was less clear for predicting variability.

Transforming between Beck Depression Inventory and CORE-OM scores in routine clinical practice.

OBJECTIVES The Clinical Outcomes in Routine Evaluation - Outcome Measure (CORE-OM) and the Beck Depression Inventory (BDI) are routinely used to assess emotional problems. It would be helpful to be able to compare scores when only one of the measures is available. We investigated the relationship between the measures and produced translation tables. METHODS Level of agreement between CORE-OM and BDI-I was assessed for 2,234 clients who had completed both measures at referral for routine secondary care. Tables for predicting between the measures were constructed using several methods, including non-linear regression and non-parametric smoothing. Results were cross-validated on a separate sample of 326 clients. RESULTS High correlations between the measures were obtained (r=.862 for female clients; r=.855 for male clients). Accuracy in predicting caseness is higher than predicting depression levels. CONCLUSIONS CORE-OM and BDI can be compared in routine clinical settings with acceptable accuracy.

Early changes, attrition, and dose-response in low intensity psychological interventions.

OBJECTIVES To investigate if early symptom changes in brief low intensity psychological interventions (guided self-help and psycho-education using cognitive behavioural therapy principles) are predictive of final treatment outcome. DESIGN Retrospective cohort data analysis. METHOD Clinical records for 1,850 patients who screened positive for depression and/or an anxiety disorder were analysed. Reliable and clinically significant improvement (RCSI) on depression (Patient Health Questionnaire-9: PHQ-9) or anxiety (generalized anxiety disorder-7: GAD-7) outcome measures after treatment was the primary outcome. Change scores ≥6 on PHQ-9 and ≥5 on GAD-7 were taken as indicative of reliable improvement (RI). The model assumed that RI in the earliest treatment sessions would be predictive of RCSI post-treatment. Predictive accuracy was assessed by calculating the area under the curve (AUC), as well as positive and negative predictive values. Diagnostic odds ratios were also estimated, adjusting for confounders such as baseline severity, use of medication, and pre-treatment symptom change. RESULTS The AUC estimates for session-to-session change scores ranged between .62 and .88, indicative of modest to high predictive reliability. Predictive accuracy was higher for patients who had four or more treatment sessions, with more than 70% of patients with RCSI being accurately identified as early as sessions 1-3. Attrition rates were significantly associated with poor outcomes. Results suggest that at least four therapy sessions are necessary to achieve more than 50% RCSI rates, and the dose-response effect appears to decline in treatments longer than six sessions. CONCLUSIONS Patients showing RI early in treatment were at least twice as likely to fully recover compared to those without early RI.

Stability of the CORE-OM and the BDI-I prior to therapy: Evidence from routine practice

BACKGROUND It is important to know the stability of standard outcome measures prior to therapy over differing periods of time that map onto the realities of waiting times in routine service settings. METHOD We studied 1,684 clients who completed one or both the targeted measures Clinical Outcomes in Routine Evaluation-Outcome Measures (CORE-OM) and Beck Depression Inventory-I (BDI-I) two times, at intervals of up to 12 months, prior to beginning psychotherapy. We also selected an additional 1,623 clients who completed the CORE-OM (N=1,623), BDI-I (N=980) or both at referral, but had no records of further contact with the service. RESULTS There was little change in the mean CORE-OM or BDI-I scores between referral and clinical assessment. The test-retest correlations showed substantial stability on both measures, declining only moderately at the longer intervals studied. CONCLUSION The high test-retest correlations for periods of up to 6 months suggest that psychological disturbance was both reliably measured by the CORE-OM and the BDI-I, and reasonably stable among clients waiting to be assessed. Implications for routine practice are discussed.

Benchmarking Routine Psychological Services: A Discussion of Challenges and Methods

BACKGROUND Policy developments in recent years have led to important changes in the level of access to evidence-based psychological treatments. Several methods have been used to investigate the effectiveness of these treatments in routine care, with different approaches to outcome definition and data analysis. AIMS To present a review of challenges and methods for the evaluation of evidence-based treatments delivered in routine mental healthcare. This is followed by a case example of a benchmarking method applied in primary care. METHOD High, average and poor performance benchmarks were calculated through a meta-analysis of published data from services working under the Improving Access to Psychological Therapies (IAPT) Programme in England. Pre-post treatment effect sizes (ES) and confidence intervals were estimated to illustrate a benchmarking method enabling services to evaluate routine clinical outcomes. RESULTS High, average and poor performance ES for routine IAPT services were estimated to be 0.91, 0.73 and 0.46 for depression (using PHQ-9) and 1.02, 0.78 and 0.52 for anxiety (using GAD-7). Data from one specific IAPT service exemplify how to evaluate and contextualize routine clinical performance against these benchmarks. CONCLUSIONS The main contribution of this report is to summarize key recommendations for the selection of an adequate set of psychometric measures, the operational definition of outcomes, and the statistical evaluation of clinical performance. A benchmarking method is also presented, which may enable a robust evaluation of clinical performance against national benchmarks. Some limitations concerned significant heterogeneity among data sources, and wide variations in ES and data completeness.

The development of a brief distress measure for the evaluation of psychotherapy and counseling (sPaCE)

Abstract An acceptable, standardized outcome measure to assess efficacy and effectiveness is needed across the professions offering psychological therapies. The goal of the current study is to present psychometric data on the reliability, validity, and sensitivity to change of the Shorter Psychotherapy and Counselling Evaluation (sPaCE) measure, a 19-item self-report instrument, with domains of Depression (general and self-harming), Anxiety (general and phobic), Apathy, and Functional Cognitive Problems. The author evaluated sPaCEs internal reliability, test–retest reliability, exploratory principal-component analysis, correlations with the Beck Depression Inventory (BDI), differences between clinical and nonclinical samples, and assessment of change within a clinical group. Internal and test–retest reliabilities were good, as was convergent validity with the BDI. A clear, replicable factor structure was demonstrated corresponding with the subscales. There were large differences between clinical and nonclinical samples and good sensitivity to change. The author concluded that the sPaCE measure is a reliable and valid instrument with good sensitivity to change and a clear factor structure and is acceptable in clinical practice.

Therapists' recall of early sudden gains in routine clinical practice

This study followed up one by Stiles et al. (2003), which identified sudden gains -- large reductions from one session to the next on a short form of clinical outcomes in routine evaluation outcome measure (CORE-SF) -- by some clients in routine clinical practice. We interviewed the therapists who had treated sudden gain and non-sudden gain clients. Results showed that therapists could retrospectively identify which clients had or had not experienced sudden CORE-SF gains at substantially better than chance rates, although they identified only about half of the clients who had experienced such gains.

A mixed-method investigation of patient monitoring and enhanced feedback in routine practice: Barriers and facilitators

Abstract Objective: To investigate the barriers and facilitators of an effective implementation of an outcome monitoring and feedback system in a UK National Health Service psychological therapy service. Method: An outcome monitoring system was introduced in two services. Enhanced feedback was given to therapists after session 4. Qualitative and quantitative methods were used, including questionnaires for therapists and patients. Thematic analysis was carried out on written and verbal feedback from therapists. Analysis of patient outcomes for 202 episodes of therapy was compared with benchmark data of 136 episodes of therapy for which feedback was not given to therapists. Results: Themes influencing the feasibility and acceptability of the feedback system were the extent to which therapists integrated the measures and feedback into the therapy, availability of administrative support, information technology, and complexity of the service. There were low levels of therapist actions resulting from the feedback, including discussing the feedback in supervision and with patients. Conclusions: The findings support the feasibility and acceptability of setting up a routine system in a complex service, but a number of challenges and barriers have to be overcome and therapist differences are apparent. More research on implementation and effectiveness is needed in diverse clinical settings.