Charles Glisson

Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challenges

One of the most critical issues in mental health services research is the gap between what is known about effective treatment and what is provided to consumers in routine care. Concerted efforts are required to advance implementation science and produce skilled implementation researchers. This paper seeks to advance implementation science in mental health services by over viewing the emergence of implementation as an issue for research, by addressing key issues of language and conceptualization, by presenting a heuristic skeleton model for the study of implementation processes, and by identifying the implications for research and training in this emerging field.

THE EFFECTS OF ORGANIZATIONAL CLIMATE AND INTERORGANIZATIONAL COORDINATION ON THE QUALITY AND OUTCOMES OF CHILDREN'S SERVICE SYSTEMS

OBJECTIVE This study examines the effects of organizational characteristics, including organizational climate and interorganizational coordination, on the quality and outcomes of childrens service systems. METHOD A quasi-experimental, longitudinal design was used to assess the effects of increasing interorganizational services coordination in public childrens service agencies. The research team collected both qualitative and quantitative data over a 3-year period describing the services provided to 250 children by 32 public childrens service offices in 24 counties in Tennessee. RESULTS Findings show that organizational climate (including low conflict, cooperation, role clarity, and personalization) is the primary predictor of positive service outcomes (the childrens improved psychosocial functioning) and a significant predictor of service quality. In contrast, interorganizational coordination had a negative effect on service quality and no effect on outcomes. CONCLUSIONS Efforts to improve public childrens service systems should focus on creating positive organizational climates rather than on increasing interorganizational services coordination. This is important because many large-scale efforts to improve childrens service systems have focused on interorganizational coordination with little success and none to date have focused on organizational climate.

The Organizational Context of Children's Mental Health Services.

This paper reviews what is known about the organizational context of childrens mental health services and describes organizational constructs, conceptual models, research methods, and intervention strategies that can be used to link organizational context to service quality and outcomes. Using evidence from studies of business and industrial organizations as well as studies of childrens service organizations, the paper presents a model of organizational effectiveness that depends on several contextual characteristics that include organizational culture, structure, climate, and work attitudes. These characteristics are believed to affect the adoption of efficacious treatments (EBPs [evidence-based practices]), adherence to treatment protocols, therapeutic alliance, and the availability, responsiveness, and continuity of services. Finally, 10 components of the ARC(Availability Responsiveness and Continuity) organizational intervention are described as examples of strategies that can be used to develop organizational contexts with the prescribed characteristics. Mental health researchers are encouraged to consider including these constructs, conceptual models, research methods, and intervention strategies in dissemination, effectiveness, and implementation studies that address the gap between research-based knowledge about mental health treatment and what is actually offered in the community.

Assessing the organizational social context (OSC) of mental health services: implications for research and practice.

The organizational social context in which mental health services are provided is believed to affect the adoption and implementation of evidence-based practices (EBPs) as well as the quality and outcomes of the services. A fully developed science of implementation effectiveness requires conceptual models that include organizational social context and tools for assessing social context that have been tested in a broad cross-section of mental health systems. This paper describes the role of organizational social context in services and implementation research and evaluates a comprehensive contextual measure, labeled Organizational Social Context (OSC), designed to assess the key latent constructs of culture, climate and work attitudes. The psychometric properties of the OSC measure were assessed in a nationwide study of 1,154 clinicians in 100 mental health clinics with a second-order confirmatory factor analysis of clinician responses, estimates of scale reliabilities, and indices of within-clinic agreement and between-clinic differences among clinicians. Finally, the paper illustrates the use of nationwide norms in describing the OSC profiles of individual mental health clinics and examines the cross-level association of organizational-level culture and climate with clinician-level work attitudes.

Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy.

OBJECTIVE A randomized trial assessed the effectiveness of a 2-level strategy for implementing evidence-based mental health treatments for delinquent youth. METHOD A 2 x 2 design encompassing 14 rural Appalachian counties included 2 factors: (a) the random assignment of delinquent youth within each county to a multisystemic therapy (MST) program or usual services and (b) the random assignment of counties to the ARC (for availability, responsiveness, and continuity) organizational intervention for implementing effective community-based mental health services. The design created 4 treatment conditions (MST plus ARC, MST only, ARC only, control). Outcome measures for 615 youth who were 69% male, 91% Caucasian, and aged 9-17 years included the Child Behavior Checklist and out-of-home placements. RESULTS A multilevel, mixed-effects, regression analysis of 6-month treatment outcomes found that youth total problem behavior in the MST plus ARC condition was at a nonclinical level and significantly lower than in other conditions. Total problem behavior was equivalent and at nonclinical levels in all conditions by the 18-month follow-up, but youth in the MST plus ARC condition entered out-of-home placements at a significantly lower rate (16%) than youth in the control condition (34%). CONCLUSIONS Two-level strategies that combine an organizational intervention such as ARC and an evidence-based treatment such as MST are promising approaches to implementing effective community-based mental health services. More research is needed to understand how such strategies can be used effectively in a variety of organizational contexts and with other types of evidence-based treatments.

Assessing and Changing Organizational Culture and Climate for Effective Services

The 2007 Aaron Rosen Lecture was presented at the annual meeting of the Society for Social Work and Research, January 12, 2007, in San Francisco, CA. The lecture begins by describing the gap between what is known about efficacious treatments and other evidence-based practices on the one hand and the services that are provided in actual community-based practice settings on the other. To address this gap, the lecture calls for the development of a science of implementation effectiveness and describes the authors research on assessing and changing the social context of mental health and social service organizations as contributing to that effort. The findings of two national studies and one controlled clinical trial are summarized to (a) link organizational social context to service outcomes, (b) describe a new profiling system for assessing organizational social context, and (c) demonstrate how social context can be changed with planned organizational intervention strategies.

Psychometric Properties and U.S. National Norms of the Evidence-Based Practice Attitude Scale (EBPAS)

The Evidence-Based Practice Attitude Scale (EBPAS) assesses mental health and social service provider attitudes toward adopting evidence-based practices. Scores on the EBPAS derive from 4 subscales (i.e., Appeal, Requirements, Openness, and Divergence) as well as the total scale, and preliminary studies have linked EBPAS scores to clinic structure and policies, organizational culture and climate, and first-level leadership. EBPAS scores are also related to service provider characteristics, including age, education level, and level of professional development. The present study examined the factor structure, reliability, and norms of EBPAS scores in a sample of 1,089 mental health service providers from a nationwide sample drawn from 100 service institutions in 26 states in the United States. The study also examined associations of provider demographic characteristics with EBPAS subscale and total scores. Confirmatory factor analysis supported a second-order factor model, and reliability coefficients for the subscales ranged from .91 to .67 (total scale = .74). The study establishes national norms for the EBPAS so that comparisons can be drawn for U.S. local as well as international studies of attitudes toward evidence-based practices. The results suggest that the factor structure and reliability are likely generalizable to a variety of service provider contexts and different service settings and that the EBPAS subscales are associated with provider characteristics. Directions for future research are discussed.

The Effects of Organizational Culture and Climate on the Access to Mental Health Care in Child Welfare and Juvenile Justice Systems

This study examined the effects of organizational culture and climate on the access to mental health care for 588 children referred to child welfare and juvenile justice systems in 21 Tennessee urban and rural counties. Cross-level, hierarchical linear models (HLM) analyses indicated that children served by child welfare and juvenile justice case management units with constructive organizational cultures were more likely to receive the needed mental health care. For example, controlling for the child’s need for mental health care and other child and family characteristics, the odds of a child receiving mental health care in a case management unit with the most constructive culture were 11 times the odds of receiving mental health care in a unit with the least constructive culture. Constructive cultures were characterized by organizational norms and expectations that case managers would be mutually supportive, develop their individual abilities, maintain positive interpersonal relationships, and be motivated to succeed. These findings suggest that efforts to improve access to mental health care for children referred to child welfare and juvenile systems should include the development of constructive organizational cultures in case management units responsible for the children’s care.

The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study

BackgroundEvidence-based practices have not been routinely adopted in community mental health organizations despite the support of scientific evidence and in some cases even legislative or regulatory action. We examined the association of clinician attitudes toward evidence-based practice with organizational culture, climate, and other characteristics in a nationally representative sample of mental health organizations in the United States.MethodsIn-person, group-administered surveys were conducted with a sample of 1,112 mental health service providers in a nationwide sample of 100 mental health service institutions in 26 states in the United States. The study examines these associations with a two-level Hierarchical Linear Modeling (HLM) analysis of responses to the Evidence-Based Practice Attitude Scale (EBPAS) at the individual clinician level as a function of the Organizational Social Context (OSC) measure at the organizational level, controlling for other organization and clinician characteristics.ResultsWe found that more proficient organizational cultures and more engaged and less stressful organizational climates were associated with positive clinician attitudes toward adopting evidence-based practice.ConclusionsThe findings suggest that organizational intervention strategies for improving the organizational social context of mental health services may contribute to the success of evidence-based practice dissemination and implementation efforts by influencing clinician attitudes.

Enhancing Schools’ Capacity to Support Children in Poverty: An Ecological Model of School-Based Mental Health Services

School based mental health services for children in poverty can capitalize on schools’ inherent capacity to support development and bridge home and neighborhood ecologies. We propose an ecological model informed by public health and organizational theories to refocus school based services in poor communities on the core function of schools to promote learning. We describe how coalescing mental health resources around school goals includes a focus on universal programming, mobilizing indigenous school and community resources, and supporting core teaching technologies. We suggest an iterative research–practice approach to program adaptation and implementation as a means toward advancing science and developing healthy children.