Gary M. Burlingame

The Reliability and Validity of the Outcome Questionnaire

With the rise in efforts to evaluate the quality of mental health care and its outcomes, the measurement of change has become an important topic. This paper tracks the creation of a new instrument designed to assess psychotherapy outcome. The Outcome Questionnaire (OQ) was designed to include items relevant to three domains central to mental health: subjective discomfort, interpersonal relations, and social role performance. This study describes the theoretical development and psychometric properties of the OQ. Psychometric properties were assessed using clinical, community, and undergraduate samples. The OQ appears to have high reliability and evidence to suggest good concurrent and construct validity of the total score. The data presented show that it distinguishes patient from non-patient samples, is sensitive to change, and correlates with other measures of patient distress.

Effectiveness of a School-Based Group Psychotherapy Program for War-Exposed Adolescents: A Randomized Controlled Trial

OBJECTIVE To evaluate the comparative effectiveness of a classroom-based psychoeducation and skills intervention (tier 1) and a school-based trauma- and grief-focused group treatment (tier 2) of a three-tiered mental health program for adolescents exposed to severe war-related trauma, traumatic bereavement, and postwar adversity. METHOD A total of 127 war-exposed and predominantly ethnic Muslim secondary school students attending 10 schools in central Bosnia who reported severe symptoms of posttraumatic stress disorder (PTSD), depression, or maladaptive grief and significant impairment in school or relationships were randomly assigned to one of two experimental conditions. These included either an active-treatment comparison condition (tier 1), consisting of a classroom-based psychoeducation and skills intervention alone (n = 61, 66% girls, mean age 16.0 years, SD 1.13) or a treatment condition composed of both the classroom-based intervention and a 17-session manual-based group therapy intervention (tier 2), trauma and grief component therapy for adolescents (n = 66, 63% girls, mean age 15.9 years, SD 1.11). Both interventions were implemented throughout the school year. Distressed students who were excluded from the study due to acute risk for harm (n = 9) were referred for community-based mental health services (tier 3). RESULTS Program effectiveness was measured via reductions in symptoms of PTSD, depression, and maladaptive grief assessed at pretreatment, posttreatment, and 4-month follow-up. Analysis of mean-level treatment effects showed significant pre- to posttreatment and posttreatment to 4-month follow-up reductions in PTSD and depression symptoms in both the treatment and comparison conditions. Significant pre- to posttreatment reductions in maladaptive grief reactions were found only in the treatment condition. Analyzed at the individual case level, the percentages of students in the treatment condition who reported significant (p <.05) pre- to posttreatment reductions in PTSD symptoms (58% at posttreatment, 81% at 4-month follow-up) compare favorably to those reported in controlled treatment efficacy trials, whereas the percentages who reported significant reductions in depression symptoms (23% at posttreatment, 61% at follow-up) are comparable to, or higher than, those found in community treatment settings. Lower but substantial percentages of significant symptom reduction were found for PTSD (33% at posttreatment, 48% at follow-up) and depression symptoms (13% at posttreatment; 47% at follow-up) in students in the comparison condition. The odds of significant symptom reduction were higher for PTSD symptoms at both posttreatment and 4-month follow-up and for maladaptive grief at posttreatment (no follow-up was conducted on maladaptive grief). Rates of significantly worsened cases were generally rare in both the treatment and comparison conditions. CONCLUSIONS A three-tiered, integrative mental health program composed of schoolwide dissemination of psychoeducation and coping skills (tier 1), specialized trauma- and grief-focused intervention for severely traumatized and traumatically bereaved youths (tier 2), and referral of youths at acute risk for community-based mental health services (tier 3) constitutes an effective and efficient method for promoting adolescent recovery in postwar settings.

Comparative Efficacy of Individual and Group Psychotherapy: A Meta-Analytic Perspective

Recent reviews of the group psychotherapy literature indicate that group is a beneficial and cost-effective treatment format. However, collective findings on the differential efficacy of group when compared with individual therapy remain problematic, incomplete, or controversial. To remedy this problem, the authors conducted a meta-analysis of 23 outcome studies that directly compared the effectiveness of the individual and group therapy formats when they were used within the same study. Results were consistent with previous reports that indicated no difference in outcome between the group and individual formats. This finding generally held true when client, therapist, methodology, treatment, and group variables were examined for possible relationship with effect sizes comparing group and individual therapy. Results bolster past findings that group therapy can be used as an efficacious cost-effective alternative to individual therapy under many different conditions.

Outcome Questionnaire: item sensitivity to change.

Although high levels of reliability are emphasized in the construction of many measures of psychological traits, tests that are intended to measure patient change following psychotherapy need to emphasize sensitivity to change as a central and primary property. This study proposes 2 criteria for evaluating the degree to which an item on a test is sensitive to change: (a) that an item changes in the theoretically proposed direction following an intervention and (b) that the change measured on an item is significantly greater in treated than in untreated individuals. Outcome Questionnaire (Lambert et al., 1996) items were subjected to item analysis by examining change rates in 284 untreated control participants and in 1,176 individuals undergoing psychotherapy. Results analyzed through multilevel or hierarchical linear modeling suggest the majority of items on this frequently used measure of psychotherapy outcome meet both criteria. Implications for test development and future research are discussed.

Assessing Clinical Significance: Proposed Extensions to Method

Jacobson, Follette, and Revenstorfs (1984) proposal for assessing clinical significance provides a needed convention for psychotherapy outcome research. Several limitations that exists in this method (Jacobson & Revenstorf, 1988) are addressed in this paper and extensions are proposed. Specifically, limitations regarding the operationalization of the underlying social validation methodology in the derivation of normative samples and the resultant standards they set are discussed. Extensions and guidelines are proposed for specifying normative samples, determining the distinctness of these samples, and expanding procedures to accommodate multiple samples. This paper initially assumes a psychometric perspective and presents extensions, based on the Symptom Checklist 90-R. Then it shifts to a clinician perspective and applies reliable change estimates and cutoff scores to actual outcome data by analyzing the progress of four patients during and after therapy. The overall merit and utility of extensions to clinical significance are then discussed.

Cohesion in group psychotherapy

Insight into the therapeutic relationship in group psychotherapy requires an understanding of the treatment context. Cohesion is defined as the therapeutic relationship in group psychotherapy emerging from the aggregate of member-leader, member-member, and member-group relationships. Using this defi

The Differential Effectiveness of Group Psychotherapy: A Meta-Analytic Perspective

The differential effectiveness of group psychotherapy was estimated in a meta-analysis of 111 experimental and quasi-experimental studies published over the past 20 years. A number of client, therapist, group, and methodological variables were examined in an attempt to determine specific as well as generic effectiveness. Three different effect sizes were computed: active versus wait list, active versus alternative treatment, and pre- to posttreatment improvement rates. The active versus wait list overall effect size (0.58) indicated that the average recipient of group treatment is better off than 72% of untreated controls. Improvement was related to group composition, setting, and diagnosis. Findings are discussed within the context of what the authors have learned about group treatment, meta-analytic studies of the extant group literature, and what remains for future research. Researchers’ understanding regarding the effectiveness of group psychotherapy has evolved over the past century. Case studies and anecdotal reports characterized the group literature in the first half of the 20th century, with the first comparative studies emerging in the 1960s (Barlow, Burlingame, & Fuhriman, 2000). Early reviews (Pattison, 1965; Rickard, 1962; Stotsky & Zolik, 1965) concluded that group therapy was a helpful adjunctive treatment, although little empirical evidence supported its use as a robust independent treatment. Reviewers in the latter part of that decade (Anderson, 1968; Mann, 1966) began to give group a heartier endorsement, describing it as capable of producing objectively measurable change in patient attitude, personality, and behavior. Throughout the 1970s, researchers repeatedly concluded that group outcomes were consistently superior to those of control groups (Bed

Group climate, cohesion, alliance, and empathy in group psychotherapy : Multilevel structural equation models

This study examined the definitional and statistical overlap among 4 key group therapeutic relationship constructs—group climate, cohesion, alliance, and empathy—across member–member, member–group, and member–leader relationships. Three multilevel structural equation models were tested using selfreport measures completed by 662 participants from 111 counseling center and personal growth groups. As hypothesized, almost all measures of therapeutic relationship were significantly correlated. Hypothesized 1-factor, 2-factor (Working and Bonding factors), and 3-factor (Member, Leader, and Group factors) models did not fit the data adequately. An exploratory model with Bonding, Working, and Negative factors provided the best fit to the data. Group members distinguished among relationships primarily according to relationship quality rather than the status or role of others (i.e., leader, member, or whole group).

Cohesion in group therapy.

Cohesion is the most popular of several relationship constructs in the clinical and empirical group therapy literature. This article reviews the most frequently cited definitions and studied measures of group cohesion. We briefly introduce a new measure, the Group Questionnaire, which elucidates group relationships by suggesting two latent factors of cohesion-relationship quality (positive bond, positive work, and negative relationship) and structure factors (member-leader and member-member). To further understand the literature, we conducted a meta-analysis examining the relationship between cohesion and treatment outcome in 40 studies. Results indicate cohesion that the weighted aggregate correlation was statistically significant with outcome r = .25, k (40), N (3,323), z = 6.54 (p < .05) with a 95% confidence interval of .17 to .32. In addition, five moderator variables were found to significantly predict the magnitude of the cohesion outcome correlation (age, theoretical orientation, length, and size of group, as well as interventions intended to enhance cohesion). Consideration of measures and practices to improve treatment outcome are highlighted.

Youth psychotherapy change trajectories and outcomes in usual care: Community mental health versus managed care settings.

OBJECTIVE The authors compared symptom change trajectories and treatment outcome categories in children and adolescents receiving routine outpatient mental health services in a public community mental health system and a private managed care organization. METHOD Archival longitudinal outcome data from parents completing the Youth Outcome Questionnaire (Y-OQ) were retrieved for children and adolescents (4-17 years old) served in a community mental health system (n = 936, mean age = 12 years, 40% girls or young women, 28% from families of color) and a managed care organization (n = 3,075, mean age = 13 years, 45% girls or young women, race and ethnicity not reported). The authors analyzed Y-OQ data using multilevel modeling and partial proportional odds modeling to test for differences in change trajectories and final outcomes across the 2 service settings. RESULTS Although initial symptom level was comparable across the 2 settings, the rate of change was significantly steeper for cases in the managed care setting. In addition, 24% of cases in the community mental health setting demonstrated a significant increase in symptoms over the course of treatment, compared with 14% of cases in the managed care setting. CONCLUSIONS These results emphasize the need for increased attention to negative outcomes in routine mental health services and provide a stronger foundation for identifying youth cases at risk for treatment failure. In addition, given the overall differences observed across treatment settings for average rate of change and deterioration rates, results suggest that setting-specific model heuristics should be used for identifying cases at risk for negative outcomes.