Nathaniel J. Williams

Assessing the Organizational Social Context (OSC) of child welfare systems: implications for research and practice.

OBJECTIVE The study: (1) provides the first assessment of the a priori measurement model and psychometric properties of the Organizational Social Context (OSC) measurement system in a US nationwide probability sample of child welfare systems; (2) illustrates the use of the OSC in constructing norm-based organizational culture and climate profiles for child welfare systems; and (3) estimates the association of child welfare system-level organizational culture and climate profiles with individual caseworker-level job satisfaction and organizational commitment. METHODS The study applies confirmatory factor analysis (CFA) and hierarchical linear models (HLM) analysis to a US nationwide sample of 1,740 caseworkers from 81 child welfare systems participating in the second National Survey of Child and Adolescent Wellbeing (NSCAW II). The participating child welfare systems were selected using a national probability procedure reflecting the number of children served by child welfare systems nationwide. RESULTS The a priori OSC measurement model is confirmed in this nationwide sample of child welfare systems. In addition, caseworker responses to the OSC scales generate acceptable to high scale reliabilities, moderate to high within-system agreement, and significant between-system differences. Caseworkers in the child welfare systems with the best organizational culture and climate profiles report higher levels of job satisfaction and organizational commitment. Organizational climates characterized by high engagement and functionality, and organizational cultures characterized by low rigidity are associated with the most positive work attitudes. CONCLUSIONS The OSC is the first valid and reliable measure of organizational culture and climate with US national norms for child welfare systems. The OSC provides a useful measure of Organizational Social Context for child welfare service improvement and implementation research efforts which include a focus on child welfare system culture and climate.

Randomized Trial of the Availability, Responsiveness, and Continuity (ARC) Organizational Intervention With Community-Based Mental Health Programs and Clinicians Serving Youth.

OBJECTIVE Evidence-based Practice (EBP) implementation is likely to be most efficient and effective in organizations with positive social contexts (i.e., organizational culture, climate, and work attitudes of clinicians). The study objective was to test whether an organizational intervention labeled Availability, Responsiveness and Continuity (ARC) could improve the organizational social contexts of community-based mental health programs for youth. METHOD The study randomly assigned 26 community-based mental health programs for youth to ARC or control conditions. The organizational cultures, climates, and work attitudes of clinicians (n = 197) in the programs were assessed with the Organizational Social Context (OSC) measure for mental health services at baseline and following the 18-month ARC intervention. RESULTS Hierarchical linear models (HLM) analyses indicated that organizational culture, climate, and work attitudes were significantly improved in the ARC condition after 18 months. Clinicians in programs assigned to ARC reported less rigid, less centralized and less apathetic organizational cultures, more engaged and functional organizational climates with less role conflict, and work attitudes with improved morale, job satisfaction, and organizational commitment. CONCLUSIONS ARC improved the organizational social contexts of clinicians in community-based mental health programs for youth. Results suggest that organizational intervention strategies can be used to create the types of organizational social contexts that are believed to be necessary for EBP implementation and other service innovations in mental health programs.

Randomized Trial of the Availability, Responsiveness and Continuity (ARC) Organizational Intervention for Improving Youth Outcomes in Community Mental Health Programs

OBJECTIVES The primary objective of the study was to assess whether the Availability, Responsiveness and Continuity (ARC) organizational intervention improved youth outcomes in community based mental health programs. The second objective was to assess whether programs with more improved organizational social contexts following the 18-month ARC intervention had better youth outcomes than programs with less improved social contexts. METHOD Eighteen community mental health programs that serve youth between the ages of 5 and 18 were randomly assigned to ARC or control conditions. Clinicians (n = 154) in the participating programs completed the Organizational Social Context (OSC) measure at baseline and following the 18-month ARC organizational intervention. Caregivers of 393 youth who were served by the 18 programs (9 in ARC and 9 in control) completed the Shortform Assessment for Children (SAC) once a month for six months beginning at intake. RESULTS Hierarchical linear models (HLM) analyses indicated that youth outcomes were significantly better in the programs that completed the 18 month ARC intervention. HLM analyses also showed that youth outcomes were best in the programs with the most improved organizational social contexts following the 18 month ARC intervention. CONCLUSIONS Youth outcomes in community mental health programs can be improved with the ARC organizational intervention and outcomes are best in programs that make the most improvements in organizational social context. The relationships linking ARC, organizational social context, and youth outcomes suggest that service improvement efforts will be more successful if those efforts include strategies to improve the organizational social contexts in which the services are embedded.

Assessing and Changing Organizational Social Contexts for Effective Mental Health Services

Culture and climate are critical dimensions of a mental health service organizations social context that affect the quality and outcomes of the services it provides and the implementation of innovations such as evidence-based treatments (EBTs). We describe a measure of culture and climate labeled Organizational Social Context (OSC), which has been associated with innovation, service quality, and outcomes in national samples and randomized controlled trials (RCTs) of mental health and social service organizations. The article also describes an empirically supported organizational intervention model labeled Availability, Responsiveness, and Continuity (ARC), which has improved organizational social context, innovation, and effectiveness in five RCTs. Finally, the article outlines a research agenda for developing more efficient and scalable organizational strategies to improve mental health services by identifying the mechanisms that link organizational interventions and social context to individual-level service provider intentions and behaviors associated with innovation and effectiveness.

Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice

A step toward the development of optimally effective, efficient, and feasible implementation strategies that increase evidence-based treatment integration in mental health services involves identification of the multilevel mechanisms through which these strategies influence implementation outcomes. This article (a) provides an orientation to, and rationale for, consideration of multilevel mediating mechanisms in implementation trials, and (b) systematically reviews randomized controlled trials that examined mediators of implementation strategies in mental health. Nine trials were located. Mediation-related methodological deficiencies were prevalent and no trials supported a hypothesized mediator. The most common reason was failure to engage the mediation target. Discussion focuses on directions to accelerate implementation strategy development in mental health.

The Organizational Social Context of Mental Health Medicaid Waiver Programs with Family Support Services: Implications for Research and Practice

Peer family support specialists (FSS) are parents with practical experience in navigating children’s mental health care systems who provide support, advocacy, and guidance to the families of children who need mental health services. Their experience and training differ from those of formally trained mental health clinicians, creating potential conflicts in priorities and values between FSS and clinicians. We hypothesized that these differences could negatively affect the organizational cultures and climates of mental health clinics that employ both FSS and mental health clinicians, and lower the job satisfaction and organizational commitment of FSS. The Organizational Social Context measure was administered on site to 209 FSS and clinicians in 21 mental health programs in New York State. The study compared the organizational-level culture and climate profiles of mental health clinics that employ both FSS and formally trained clinicians to national norms for child mental health clinics, assessed individual-level job satisfaction and organizational commitment as a function of job (FSS vs. clinician) and other individual-level and organizational-level characteristics, and tested whether FSS and clinicians job attitudes were differentially associated with organizational culture and climate. The programs organizational culture and climate profiles were not significantly different from national norms. Individual-level job satisfaction and organizational commitment were unrelated to position (FSS vs. clinician) or other individual-level and organizational-level characteristics except for culture and climate. Both FSS’ and clinicians’ individual-level work attitudes were associated similarly with organizational culture and climate.

Oh How I Try to Use Evidence in my Social Work Practice: Efforts, Successes, Frustrations, and Questions

Convinced by the literature that they could do better for their clients, in this article the authors describe their efforts to transform a categorically-funded program model at one agency from relying on authority and good intentions to evidence-based practice (EBP). Using a case study approach the authors describe a six-year research effort in which they experienced a modicum of success, including completion of a few projects, followed by paralyzing frustrations that warrant sharing with other social work professionals. Reflecting on their experiences, they posit a few questions about EBP for practitioners and leaders in the profession to contemplate. The authors then offer five pathways for moving the profession closer to making EBP a reality in social work.

Concordance Between Administrator and Clinician Ratings of Organizational Culture and Climate

Organizational culture and climate are important determinants of behavioral health service delivery for youth. The Organizational Social Context measure is a well validated assessment of organizational culture and climate that has been developed and extensively used in public sector behavioral health service settings. The degree of concordance between administrators and clinicians in their reports of organizational culture and climate may have implications for research design, inferences, and organizational intervention. However, the extent to which administrators’ and clinicians’ reports demonstrate concordance is just beginning to garner attention in public behavioral health settings in the United States. We investigated the concordance between 73 administrators (i.e., supervisors, clinical directors, and executive directors) and 247 clinicians in 28 child-serving programs in a public behavioral health system. Findings suggest that administrators, compared to clinicians, reported more positive cultures and climates. Organizational size moderated this relationship such that administrators in small programs (<466 youth clients served annually) provided more congruent reports of culture and climate in contrast to administrators in large programs (≥466 youth clients served annually) who reported more positive cultures and climates than clinicians. We propose a research agenda that examines the effect of concordance between administrators and clinicians on organizational outcomes in public behavioral health service settings.

The relationship between consumer, clinician, and organizational characteristics and use of evidence-based and non-evidence-based therapy strategies in a public mental health system

We investigated the relationship between consumer, clinician, and organizational factors and clinician use of therapy strategies within a system-wide effort to increase the use of cognitive-behavioral therapy. Data from 247 clinicians in 28 child-serving organizations were collected. Clinicians participating in evidence-based practice training initiatives were more likely to report using cognitive-behavioral therapy when they endorsed more clinical experience, being salaried clinicians, and more openness to evidence-based practice. Clinicians participating in evidence-based practice initiatives were more likely to use psychodynamic techniques when they had older clients, less knowledge about evidence-based practice, more divergent attitudes toward EBP, higher financial strain, and worked in larger organizations. In clinicians not participating in evidence-based training initiatives; depersonalization was associated with higher use of cognitive-behavioral; whereas clinicians with less knowledge of evidence-based practices were more likely to use psychodynamic techniques. This study suggests that clinician characteristics are important when implementing evidence-based practices; and that consumer, clinician, and organizational characteristics are important when de-implementing non evidence-based practices. This work posits potential characteristics at multiple levels to target with implementation and deimplementation strategies.

Reliability and validity of a treatment adherence measure for child psychiatric rehabilitation.

OBJECTIVE Treatment adherence, defined as the degree to which practitioners implemented prescribed program principles and activities and avoided proscribed activities, has been an area of growing interest in mental health services for children with severe emotional and behavioral disorders. This study evaluated the reliability and validity of a treatment adherence measure for child psychiatric rehabilitation (CPSR). METHODS Parents of children receiving CPSR (n = 79) or psychotherapy (n = 27) completed the Childrens Psychosocial Rehabilitation Treatment Adherence Measure (CTAM) and a measure of 2-week session impact. Psychiatric rehabilitation (PSR) supervisors identified PSR practitioners with reputations for high or low adherence to the model. The CTAMs discriminant validity was assessed by using known-groups procedures and predictive validity by examining its relationship to 2-week session impact. RESULTS The CTAM demonstrated excellent internal consistency (α = .92), discriminant validity (p = .002, d = .72; p = .021, d = .59), and predictive validity (B = 2.24, SE = .31, p < .001), accounting for 28% of the child-level variance in 2-week session impact. CONCLUSIONS AND IMPLICATIONS FOR PRACTICE Findings suggest the CTAM is a reliable and valid measure of treatment adherence for CPSR programs with a skill-teaching focus. Providers and agencies should take steps to enhance treatment adherence because it may be an important predictor of childrens short-term response to CPSR.