Ana M. Ugueto

Performance of Evidence-Based Youth Psychotherapies Compared With Usual Clinical Care A Multilevel Meta-analysis

IMPORTANCE Research across more than 4 decades has produced numerous empirically tested evidence-based psychotherapies (EBPs) for psychopathology in children and adolescents. The EBPs were developed to improve on usual clinical interventions. Advocates argue that the EBPs should replace usual care, but this assumes that EBPs produce better outcomes than usual care. OBJECTIVE To determine whether EBPs do in fact produce better outcomes than usual care in youth psychotherapy. We performed a meta-analysis of 52 randomized trials directly comparing EBPs with usual care. Analyses assessed the overall effect of EBPs vs usual care and candidate moderators; we used multilevel analysis to address the dependency among effect sizes (ES) that is common but typically unaddressed in psychotherapy syntheses. DATA SOURCES We searched the PubMed, PsychINFO, and Dissertation Abstracts International databases for studies from January 1, 1960, through December 31, 2010. STUDY SELECTION We identified 507 randomized youth psychotherapy trials. Of these, the 52 studies that compared EBPs with usual care were included in the meta-analysis. DATA EXTRACTION AND SYNTHESIS Sixteen variables (participant, treatment, outcome, and study characteristics) were extracted from studies, and ESs were calculated for all comparisons of EBP vs usual care. We used an extension of the commonly used random-effects meta-analytic model to obtain an overall estimate of the difference between EBP and usual care while accounting for the dependency among ESs. We then fitted a 3-level mixed-effects model to identify moderators that might explain variation in ESs within and between studies by adding study or ES characteristics as fixed predictors. MAIN OUTCOMES AND MEASURES Primary outcomes of our meta-analysis were mean ES estimates across all studies and for levels of candidate moderators. These ES values were based on measures of symptoms, functioning, and other outcomes assessed within the 52 randomized trials. RESULTS Evidence-based psychotherapies outperformed usual care. Mean ES was 0.29; the probability was 58% that a randomly selected youth would have a better outcome after EBP than a randomly selected youth after receiving usual care. The following 3 variables moderated treatment benefit: ESs decreased for studies conducted outside North America, for studies in which all participants were impaired enough to qualify for diagnoses, and for outcomes reported by informants other than the youths and parents in therapy. For certain key groups (eg, studies of clinically referred samples and youths with diagnoses), significant EBP effects were not demonstrated. CONCLUSIONS AND RELEVANCE Evidence-based psychotherapies outperform usual care, but the EBP advantage is modest and moderated by youth, location, and assessment characteristics. The EBPs have room for improvement in the magnitude and range of their benefit relative to usual clinical care.

Youth Top Problems: using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy.

OBJECTIVE To complement standardized measurement of symptoms, we developed and tested an efficient strategy for identifying (before treatment) and repeatedly assessing (during treatment) the problems identified as most important by caregivers and youths in psychotherapy. METHOD A total of 178 outpatient-referred youths, 7-13 years of age, and their caregivers separately identified the 3 problems of greatest concern to them at pretreatment and then rated the severity of those problems weekly during treatment. The Top Problems measure thus formed was evaluated for (a) whether it added to the information obtained through empirically derived standardized measures (e.g., the Child Behavior Checklist [CBCL; Achenbach & Rescorla, 2001] and the Youth Self-Report [YSR; Achenbach & Rescorla, 2001]) and (b) whether it met conventional psychometric standards. RESULTS The problems identified were significant and clinically relevant; most matched CBCL/YSR items while adding specificity. The top problems also complemented the information yield of the CBCL/YSR; for example, for 41% of caregivers and 79% of youths, the identified top problems did not correspond to any items of any narrowband scales in the clinical range. Evidence on test-retest reliability, convergent and discriminant validity, sensitivity to change, slope reliability, and the association of Top Problems slopes with standardized measure slopes supported the psychometric strength of the measure. CONCLUSIONS The Top Problems measure appears to be a psychometrically sound, client-guided approach that complements empirically derived standardized assessment; the approach can help focus attention and treatment planning on the problems that youths and caregivers consider most important and can generate evidence on trajectories of change in those problems during treatment.

Evidence-Based Youth Psychotherapy in the Mental Health Ecosystem

Five decades of randomized trials research have produced dozens of evidence-based psychotherapies (EBPs) for youths. The EBPs produce respectable effects in traditional efficacy trials, but the effects shrink markedly when EBPs are tested in practice contexts with clinically referred youths and compared to usual clinical care. We considered why this might be the case. We examined relevant research literature and drew examples from our own research in practice settings. One reason for the falloff in EBP effects may be that so little youth treatment research has been done in the context of everyday practice. Researchers may have missed opportunities to learn how to make EBPs work well in the actual youth mental health ecosystem, in which so many real-world factors are at play that cannot be controlled experimentally. We sketch components and characteristics of that ecosystem, including clinically referred youths, their caregivers and families, the practitioners who provide their care, the organizations within which care is provided, the network of youth service systems (e.g., child welfare, education), and the policy context (e.g., reimbursement regulations and incentives). We suggest six strategies for future research on EBPs within the youth mental health ecosystem, including reliance on the deployment-focused model of development and testing, testing the mettle of current EBPs in everyday practice contexts, using the heuristic potential of usual care, testing restructured and integrative adaptations of EBPs, studying the use of treatment response feedback to guide clinical care, and testing models of the relation between policy change and EBP implementation.

What five decades of research tells us about the effects of youth psychological therapy: A multilevel meta-analysis and implications for science and practice

Across 5 decades, hundreds of randomized trials have tested psychological therapies for youth internalizing (anxiety, depression) and externalizing (misconduct, attention deficit and hyperactivity disorder) disorders and problems. Since the last broad-based youth meta-analysis in 1995, the number of trials has almost tripled and data-analytic methods have been refined. We applied these methods to the expanded study pool (447 studies; 30,431 youths), synthesizing 50 years of findings and identifying implications for research and practice. We assessed overall effect size (ES) and moderator effects using multilevel modeling to address ES dependency that is common, but typically not modeled, in meta-analyses. Mean posttreatment ES was 0.46; the probability that a youth in the treatment condition would fare better than a youth in the control condition was 63%. Effects varied according to multiple moderators, including the problem targeted in treatment: Mean ES at posttreatment was strongest for anxiety (0.61), weakest for depression (0.29), and nonsignificant for multiproblem treatment (0.15). ESs differed across control conditions, with “usual care” emerging as a potent comparison condition, and across informants, highlighting the need to obtain and integrate multiple perspectives on outcome. Effects of therapy type varied by informant; only youth-focused behavioral therapies (including cognitive-behavioral therapy) showed similar and robust effects across youth, parent, and teacher reports. Effects did not differ for Caucasian versus minority samples, but more diverse samples are needed. The findings underscore the benefits of psychological treatments as well as the need for improved therapies and more representative, informative, and rigorous intervention science.

Modular Approach to Therapy for Anxiety, Depression, Trauma, or Conduct Problems in outpatient child and adolescent mental health services in New Zealand: study protocol for a randomized controlled trial

BackgroundMental health disorders are common and disabling for young people because of the potential to disrupt key developmental tasks. Implementation of evidence-based psychosocial therapies in New Zealand is limited, owing to the inaccessibility, length, and cost of training in these therapies. Furthermore, most therapies address one problem area at a time, although comorbidity and changing clinical needs commonly occur in practice. A more flexible approach is needed. The Modular Approach to Therapy for Children with Anxiety, Depression, Trauma, or Conduct Problems (MATCH-ADTC) is designed to overcome these challenges; it provides a range of treatment modules addressing different problems, within a single training program. A clinical trial of MATCH-ADTC in the USA showed that MATCH-ADTC outperformed usual care and standard evidence-based treatment on several clinical measures. We aim to replicate these findings and evaluate the impact of providing training and supervision in MATCH-ADTC to: (1) improve clinical outcomes for youth attending mental health services; (2) increase the amount of evidence-based therapy content; (3) increase the efficiency of service delivery.MethodsThis is an assessor-blinded multi-site effectiveness randomized controlled trial. Randomization occurs at two levels: (1) clinicians (≥60) are randomized to intervention or usual care; (2) youth participants (7–14 years old) accepted for treatment in child and adolescent mental health services (with a primary disorder that includes anxiety, depression, trauma-related symptoms, or disruptive behavior) are randomly allocated to receive MATCH-ADTC or usual care. Youth participants are recruited from ‘mainstream’, Māori-specific, and Pacific-specific child and adolescent mental health services. We originally planned to recruit 400 youth participants, but this has been revised to 200 participants. Centralized computer randomization ensures allocation concealment. The primary outcome measures are: (i) the difference in trajectory of change of clinical severity between groups (using the parent-rated Brief Problem Monitor); (ii) clinicians’ use of evidence-based treatment procedures during therapy sessions; (iii) total time spent by clinicians delivering therapy.DiscussionIf MATCH-ADTC demonstrates effectiveness it could offer a practical efficient method to increase access to evidence-based therapies, and improve outcomes for youth attending secondary care services.Trial registrationAustralian and New Zealand Clinical Trials Registry ACTRN12614000297628.

When the torch is passed, does the flame still burn? Testing a “train the supervisor” model for the Child STEPs treatment program.

Objective: We assessed sustainability of an empirically supported, transdiagnostic youth psychotherapy program when therapist supervision was shifted from external experts to internal clinic staff. Method: One hundred sixty-eight youths, aged 6–15 years, 59.5% male, 85.1% Caucasian, were treated for anxiety, depression, traumatic stress, or conduct problems by clinicians employed in community mental health clinics. In Phase 1 (2.7 years), 1 group of clinicians, the Sustain group, received training in Child STEPs (a modular transdiagnostic treatment + weekly feedback on youth response) and treated clinic-referred youths, guided by weekly supervision from external STEPs experts. In Phase 2 (2.9 years), Sustain clinicians treated additional youths but with supervision by clinic staff who had been trained to supervise STEPs. Also in Phase 2, a new group, External Supervision clinicians, received training and supervision from external STEPs experts and treated referred youths. Phase 2 youths were randomized to Sustain or External Supervision clinicians. Groups were compared on 3 therapist fidelity measures and 14 clinical outcome measures. Results: Sustain clinicians maintained their previous levels of fidelity and youth outcomes after switching from external to internal supervision; and in Phase 2, the Sustain and External Supervision groups also did not differ on fidelity or youth outcomes. Whereas all 34 group comparisons were nonsignificant, trends with the largest effect sizes showed better clinical outcomes for internal than external supervision. Conclusions: Implementation of empirically supported transdiagnostic treatment may be sustained when supervision is transferred from external experts to trained clinic staff, potentially enhancing cost-effectiveness and staying power in clinical practice.