Michael S. Hurlburt

Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors

Implementation science is a quickly growing discipline. Lessons learned from business and medical settings are being applied but it is unclear how well they translate to settings with different historical origins and customs (e.g., public mental health, social service, alcohol/drug sectors). The purpose of this paper is to propose a multi-level, four phase model of the implementation process (i.e., Exploration, Adoption/Preparation, Implementation, Sustainment), derived from extant literature, and apply it to public sector services. We highlight features of the model likely to be particularly important in each phase, while considering the outer and inner contexts (i.e., levels) of public sector service systems.

Parent-Training Programs in Child Welfare Services: Planning for a More Evidence-Based Approach to Serving Biological Parents.

Child welfare service agencies provide parent training as part of their legally mandated responsibility to provide services to assist families to keep their children at home or to achieve reunification. The use of parent-training programs for families in the child welfare system has undergone relatively little examination. Mental health, special education, and juvenile justice have been identifying evidence-based approaches that have demonstrated effectiveness with children and families with conduct disorders and other behavioral problems, although few of these interventions have been tested with child welfare services clientele. This article brings together evidence about the most promising programs from other child service sectors with information about the current parent training approaches in child welfare and generates a range of proposals about next steps to enhance the capacity of parent training and fulfill the high expectations set in law and practice.

Developmental and behavioral needs and service use for young children in child welfare.

Objective. To determine the level of developmental and behavioral need in young children entering child welfare (CW), estimate early intervention services use, and examine variation in need and service use based on age and level of involvement with CW by using a national probability sample in the United States. Methods. As part of the National Survey of Child and Adolescent Well-Being, data were collected on 2813 children <6 years old for whom possible abuse or neglect was investigated by CW agencies. Analyses used descriptive statistics to determine developmental and behavioral needs across 5 domains (cognition, behavior, communication, social, and adaptive functioning) and service use. Logistic regression was used to examine the relationship between independent variables (age, gender, race-ethnicity, maltreatment history, level of CW involvement, and developmental or behavior problems) and service use. Results. Results indicate that age and level of CW involvement predict service use when controlling for need. Both toddlers (41.8%) and preschoolers (68.1%) in CW have high developmental and behavioral needs; however, few children are receiving services for these issues (22.7% overall). Children that remain with their biological parents have similar needs to those in out-of-home care but are less likely to use services. Children <3 years of age are least likely to use services. Conclusions. Children referred to CW have high developmental and behavioral need regardless of the level of CW involvement. Both age and level of involvement influence service use when controlling for need. Mechanisms need to be developed to address disparities in access to intervention.

Mental Health Care for Children With Disruptive Behavior Problems: A View Inside Therapists' Offices

OBJECTIVES In the United States, more money is spent on treatment for childrens mental health problems than for any other childhood medical condition, yet little is known about usual care treatment for children. Objectives of this study were to characterize usual care outpatient psychotherapy for children with disruptive behavior problems and to identify consistencies and inconsistencies between usual care and common elements of evidence-based practices in order to inform efforts to implement evidence-based practices in usual care. METHODS Participants included 96 psychotherapists and 191 children aged four to 13 who were presenting for treatment for disruptive behavior to one of six usual care clinics. An adapted version of the Therapy Process Observational Coding System for Child Psychotherapy-Strategies scale (TPOCS-S) was used to assess psychotherapy processes in 1,215 randomly selected (out of 3,241 collected) videotaped treatment sessions; treatment sessions were recorded for up to 16 months. RESULTS Most children received a large amount of treatment (mean number of sessions=22, plus children received other auxiliary services), and there was great variability in the amount and type of care received. Therapists employed a wide array of treatment strategies directed toward children and parents within and across sessions, but on average all strategies were delivered at a low intensity. Several strategies that were conceptually consistent with evidence-based practices were observed frequently (for example, affect education and using positive reinforcement); however, others were observed rarely (for example, assigning or reviewing homework and role-playing). CONCLUSIONS Usual care treatment for these youths reflected great breadth but not depth. The results highlight specific discrepancies between evidence-based care and usual care, thus identifying potentially potent targets for improving the effectiveness of usual care.

Quality of Publicly-Funded Outpatient Specialty Mental Health Care for Common Childhood Psychiatric Disorders in California

OBJECTIVE To describe the documented adherence to quality indicators for the outpatient care of attention-deficit/hyperactivity disorder, conduct disorder, and major depression for children in public mental health clinics and to explore how adherence varies by child and clinic characteristics. METHOD A statewide, longitudinal cohort study of 813 children ages 6.0-16.9 years with at least 3 months of outpatient care, drawn from 4,958 patients in 62 mental health clinics in California from August 1, 1998, through May 31, 1999. The main outcome was documented adherence to quality indicators based on scientific evidence and clinical judgment, assessed by explicit medical record review. RESULTS Relatively high adherence was recorded for clinical assessment (78%-95%), but documented adherence to quality indicators related to service linkage, parental involvement, use of evidence-based psychosocial treatment, and patient protection were moderate to poor (74.1%-8.0%). For children prescribed psychotropic medication, 28.3% of the records documented monitoring of at least one clinically indicated vital sign or laboratory study. Documented adherence to quality indicators varied little by child demographics or clinic factors. CONCLUSION Efforts to improve care should be directed broadly across clinics, with documentation of safe practices, particularly for children prescribed psychotropic medication, being of highest priority.

Children in Out-of-Home Care: Entry Into Intensive or Restrictive Mental Health and Residential Care Placements

Using longitudinal data from the National Survey on Child and Adolescent Well-Being (NSCAW), this study investigates entry into intensive or restrictive settings during a 36-month study period. Specifically, this analysis examines entry into treatment foster care, group homes, residential treatment, and inpatient psychiatric care for youth placed into out-of-home care (n = 981). It aims to determine at what point in their first out-of-home episodes and for what reasons youth entered such settings. As NSCAW used a national probability sampling design,this analysis provides national estimates about entry into intensive or restrictive settings for youth in out-of-home care. Twenty-five percent of youth (n = 280) experienced an intensive or restrictive setting during their first out-of-home care episode; 70% were in either group homes (33.2%) or residential treatment settings (37.0%).About half of the youth with such placements (48.9%) were placed into intensive or restrictive settings as a first placement during their first out-of-home episode.

Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation

BackgroundLeadership is important in the implementation of innovation in business, health, and allied health care settings. Yet there is a need for empirically validated organizational interventions for coordinated leadership and organizational development strategies to facilitate effective evidence-based practice (EBP) implementation. This paper describes the initial feasibility, acceptability, and perceived utility of the Leadership and Organizational Change for Implementation (LOCI) intervention. A transdisciplinary team of investigators and community stakeholders worked together to develop and test a leadership and organizational strategy to promote effective leadership for implementing EBPs.MethodsParticipants were 12 mental health service team leaders and their staff (n = 100) from three different agencies that provide mental health services to children and families in California, USA. Supervisors were randomly assigned to the 6-month LOCI intervention or to a two-session leadership webinar control condition provided by a well-known leadership training organization. We utilized mixed methods with quantitative surveys and qualitative data collected via surveys and a focus group with LOCI trainees.ResultsQuantitative and qualitative analyses support the LOCI training and organizational strategy intervention in regard to feasibility, acceptability, and perceived utility, as well as impact on leader and supervisee-rated outcomes.ConclusionsThe LOCI leadership and organizational change for implementation intervention is a feasible and acceptable strategy that has utility to improve staff-rated leadership for EBP implementation. Further studies are needed to conduct rigorous tests of the proximal and distal impacts of LOCI on leader behaviors, implementation leadership, organizational context, and implementation outcomes. The results of this study suggest that LOCI may be a viable strategy to support organizations in preparing for the implementation and sustainment of EBP.

Comparing DISC-IV and Clinician Diagnoses Among Youths Receiving Public Mental Health Services

OBJECTIVES To compare the prevalence and agreement of diagnoses based on Diagnostic Interview Schedule for Children Version IV (DISC-IV) and clinician assignment for youths receiving public mental health services between 1996 and 1997 and to examine potential predictors of diagnostic agreement. METHOD Participants included 240 youths aged 6-18 years. Past-year prevalence rates and kappa statistics were calculated for four diagnostic categories: anxiety, mood, attention-deficit/hyperactivity disorder (ADHD), and disruptive behavior disorders (DBD). Potential predictors of diagnostic agreement were examined with logistic regression analysis. RESULTS The prevalence of ADHD, DBD, and anxiety disorders was significantly higher based on the DISC-IV, while the prevalence of mood disorders was significantly higher based on clinician assignment. Diagnostic agreement was poor overall. The kappa values ranged from -0.04 for anxiety disorders to 0.22 for ADHD. Significant predictors of agreement varied by diagnosis and included symptom severity, comorbidity, youth age and gender, and school-based problem identification. CONCLUSIONS Consistent with previous findings of poor diagnostic agreement between structured interviews and clinicians, these results call for a better understanding of factors affecting diagnostic assignment across different methods. This is especially important if researchers continue to use structured interviews to determine prevalence, establish diagnosis-based treatment guidelines, and disseminate evidence-based treatments to community mental health settings.

Mixed-Methods Designs in Mental Health Services Research: A Review

OBJECTIVE Despite increased calls for use of mixed-methods designs in mental health services research, how and why such methods are being used and whether there are any consistent patterns that might indicate a consensus about how such methods can and should be used are unclear. METHODS Use of mixed methods was examined in 50 peer-reviewed journal articles found by searching PubMed Central and 60 National Institutes of Health (NIH)-funded projects found by searching the CRISP database over five years (2005-2009). Studies were coded for aims and the rationale, structure, function, and process for using mixed methods. RESULTS A notable increase was observed in articles published and grants funded over the study period. However, most did not provide an explicit rationale for using mixed methods, and 74% gave priority to use of quantitative methods. Mixed methods were used to accomplish five distinct types of study aims (assess needs for services, examine existing services, develop new or adapt existing services, evaluate services in randomized controlled trials, and examine service implementation), with three categories of rationale, seven structural arrangements based on timing and weighting of methods, five functions of mixed methods, and three ways of linking quantitative and qualitative data. Each study aim was associated with a specific pattern of use of mixed methods, and four common patterns were identified. CONCLUSIONS These studies offer guidance for continued progress in integrating qualitative and quantitative methods in mental health services research consistent with efforts by NIH and other funding agencies to promote their use.

Mental Health Problems in Young Children Investigated by U.S. Child Welfare Agencies

OBJECTIVE To examine the prevalence/predictors of mental health (MH) problems and services use in 12- to 36-month-old children who had been investigated for maltreatment. METHOD Data came from the second National Survey of Child and Adolescent Well-Being (NSCAW II), a longitudinal study of youth ages 0 to 17.5 years referred to U.S. child welfare agencies. These analyses involved 1117 children 12 to 36 months of age. Sociodemographic, social services, developmental and health data were collected on the children and caregivers. Outcomes were scores over the clinical cutoffs on the Brief Infant Toddler Social and Emotional Assessment (BITSEA) Scales for 12- to 18-month-olds and the Child Behavior Checklist (CBCL) for 19- to 36-month-olds. RESULTS In all, 34.6% of 12 to 18 month-olds scored high on the Problem Scale of the BITSEA, and 20.9% on the Competence Scale, whereas 10.0% of 19- to 36-month-olds scored over the CBCL clinical cut-off. Children of black ethnicity were less likely to have elevated scores on the BITSEA Problem Scale, whereas children who lived with a never-married caregiver were five times more likely to have elevated scores. Competence problems were associated with prior child welfare history. Elevated CBCL scores were associated with living with a depressed caregiver. Few children with identified MH problems, 2.2%, received an MH service. When we added parenting skills training that might be related to the treatment of child problems, 19.2% received a service. CONCLUSIONS Identifiable MH problems are common, but few children receive services for those problems. The lack of services received by these young, multi-challenged children is a services systems and social policy failure.