Debra Hecht

The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: evidence for a protective effect.

Staff retention is an ongoing challenge in mental health and community-based service organizations. Little is known about the impact of evidence-based practice implementation on the mental health and social service workforce. The present study examined the effect of evidence-based practice implementation and ongoing fidelity monitoring on staff retention in a childrens services system. The study took place in the context of a statewide, regionally randomized effectiveness trial of an evidence-based intervention designed to reduce child neglect. In the study 21 teams consisting of 153 home-based service providers were followed over a 29-month period. Survival analyses revealed greater staff retention in the condition where the evidence-based practice was implemented along with ongoing fidelity monitoring presented to staff as supportive consultation. These results should help to allay concerns about staff retention when implementing evidence-based practices where there is good values-innovation fit and when fidelity monitoring is designed as an aid and support to service providers in providing a high standard of care for children and families.

A Statewide Trial of the SafeCare Home-based Services Model With Parents in Child Protective Services

OBJECTIVES: In this trial, we compared Child Protective Services (CPS) recidivism outcomes between the home-based SafeCare (SC) model for child neglect and comparable home-based services, but without SC modules, for parents in the CPS system across 2 quality control strategies: coached (C) and uncoached implementation. SC is a home-based behavioral skills training model designed for neglecting or maltreating parents. The study was conducted in a scaled-up, statewide implementation setting. METHODS: Two thousand one hundred seventy-five maltreating parents, treated by 219 home visitors, were enrolled and treated in a 2 × 2 (SC versus services as usual × C versus uncoached implementation strategy) randomized cluster experiment. Cases were followed for an average of 6 years for CPS recidivism events. Subpopulation analyses were conducted for parents meeting customary SC inclusion criteria. RESULTS: Consistently significant main effects in favor of SC were found across simple and more complex modeling approaches (hazard ratios = 0.74–0.83). Larger effects were found among the subpopulation meeting customary SC inclusion criteria. C implementation yielded smaller and occasionally significant effects in analyses that included more diverse cases falling outside customary SC inclusion criteria. CONCLUSIONS: Findings support the adoption and use of SC within CPS home-based services systems. C implementation may be especially valuable for cases where the client-model fit is less strong.

Dynamic adaptation process to implement an evidence-based child maltreatment intervention

BackgroundAdaptations are often made to evidence-based practices (EBPs) by systems, organizations, and/or service providers in the implementation process. The degree to which core elements of an EBP can be maintained while allowing for local adaptation is unclear. In addition, adaptations may also be needed at the system, policy, or organizational levels to facilitate EBP implementation and sustainment. This paper describes a study of the feasibility and acceptability of an implementation approach, the Dynamic Adaptation Process (DAP), designed to allow for EBP adaptation and system and organizational adaptations in a planned and considered, rather than ad hoc, way. The DAP involves identifying core elements and adaptable characteristics of an EBP, then supporting implementation with specific training on allowable adaptations to the model, fidelity monitoring and support, and identifying the need for and solutions to system and organizational adaptations. In addition, this study addresses a secondary concern, that of improving EBP model fidelity assessment and feedback in real-world settings.MethodsThis project examines the feasibility, acceptability, and utility of the DAP; tests the degree to which fidelity can be maintained using the DAP compared to implementation as usual (IAU); and examines the feasibility of using automated phone or internet-enabled, computer-based technology to assess intervention fidelity and client satisfaction. The study design incorporates mixed methods in order to describe processes and factors associated with variations in both how the DAP itself is implemented and how the DAP impacts fidelity, drift, and adaptation. The DAP model is to be examined by assigning six regions in California (USA) to either the DAP (n = 3) or IAU (n = 3) to implement an EBP to prevent child neglect.DiscussionThe DAP represents a data-informed, collaborative, multiple stakeholder approach to maintain intervention fidelity during the implementation of EBPs in the field by providing support for intervention, system, and organizational adaptation and intervention fidelity to meet local needs. This study is designed to address the real-world implications of EBP implementation in public sector service systems and is relevant for national, state, and local service systems and organizations.

Treatment for preschool children with interpersonal sexual behavior problems: a pilot study.

This pilot study evaluated a 12-week group treatment program for preschool children with interpersonal sexual behavior problems (SBP; N = 85; 53 completed at least 8 sessions). Many children presented with co-occurring trauma symptoms and disruptive behaviors. In intent-to-treat analysis, a significant linear reduction in SBP due to number of treatment sessions attended was found, an effect that was independent of linear reductions affiliated with elapsed time. Under the assumption that treatment can have an incremental impact, more than one third of the variance was accounted for by treatment effects, with female and older children most favorably impacted. Caregivers reported increase in knowledge, satisfaction, and usefulness of treatment. In addition to replication, future research is needed to examine (a) effects of environment change and time on SBP, (b) stability of treatment effects, and (c) best practices to integrate evidence-based treatments for comorbid conditions.

Mixed-method study of a conceptual model of evidence-based intervention sustainment across multiple public-sector service settings.

BackgroundThis study examines sustainment of an EBI implemented in 11 United States service systems across two states, and delivered in 87 counties. The aims are to 1) determine the impact of state and county policies and contracting on EBI provision and sustainment; 2) investigate the role of public, private, and academic relationships and collaboration in long-term EBI sustainment; 3) assess organizational and provider factors that affect EBI reach/penetration, fidelity, and organizational sustainment climate; and 4) integrate findings through a collaborative process involving the investigative team, consultants, and system and community-based organization (CBO) stakeholders in order to further develop and refine a conceptual model of sustainment to guide future research and provide a resource for service systems to prepare for sustainment as the ultimate goal of the implementation process.MethodsA mixed-method prospective and retrospective design will be used. Semi-structured individual and group interviews will be used to collect information regarding influences on EBI sustainment including policies, attitudes, and practices; organizational factors and external policies affecting model implementation; involvement of or collaboration with other stakeholders; and outer- and inner-contextual supports that facilitate ongoing EBI sustainment. Document review (e.g., legislation, executive orders, regulations, monitoring data, annual reports, agendas and meeting minutes) will be used to examine the roles of state, county, and local policies in EBI sustainment. Quantitative measures will be collected via administrative data and web surveys to assess EBI reach/penetration, staff turnover, EBI model fidelity, organizational culture and climate, work attitudes, implementation leadership, sustainment climate, attitudes toward EBIs, program sustainment, and level of institutionalization. Hierarchical linear modeling will be used for quantitative analyses. Qualitative analyses will be tailored to each of the qualitative methods (e.g., document review, interviews). Qualitative and quantitative approaches will be integrated through an inclusive process that values stakeholder perspectives.DiscussionThe study of sustainment is critical to capitalizing on and benefiting from the time and fiscal investments in EBI implementation. Sustainment is also critical to realizing broad public health impact of EBI implementation. The present study takes a comprehensive mixed-method approach to understanding sustainment and refining a conceptual model of sustainment.

Change Trajectories During Home Based Services With Chronic Child Welfare Cases

This study examines how risk factor change patterns vary with case chronicity, and whether risk factor improvement still predicts lower recidivism risk among chronic cases. 2,175 parents in home based child welfare services were surveyed for risk factors at pre-treatment, post-treatment and 6-month follow-up. Mixture modeling of latent difference scores identified change trajectory classes related retrospectively to chronicity and prospectively to recidivism. Five change trajectories were identified: stable low problem, stable high problem, sustained improvement, relapsing, and paradoxical. Chronicity was associated with a decreasing probability of membership in the stable low problem trajectory and increasing probability of membership in the stable high problem and sustained improvement trajectories. Cases with more favorable trajectories recidivated less across levels of chronicity. Findings suggest that chronic cases may improve little, but still retain a stable or increasing chance of sustained improvement associated with lower risk. A cumulative service benefit might be one possible explanation for this observation, and might suggest that repeated intervention efforts are not always wasted on chronic cases. The current episodic and reactive service delivery model in child welfare may be a mismatch with chronic cases where progress is absent or tends to occur cumulatively across service episodes.

Contracting and Procurement for Evidence-Based Interventions in Public-Sector Human Services: A Case Study

Sustainment of evidence-based interventions (EBIs) in human services depends on the inner context of community-based organizations (CBOs) that provide services and the outer context of their broader environment. Increasingly, public officials are experimenting with contracting models from for-profit industries to procure human services. In this case study, we conducted qualitative interviews with key government and CBO stakeholders to examine implementation of the Best Value-Performance Information Procurement System to contract for EBIs in a child welfare system. Findings suggest that stakeholder relationships may be compromised when procurement disregards local knowledge, communication, collaboration, and other factors supporting EBIs and public health initiatives.

Mixed-methods feasibility study on the cultural adaptation of a child abuse prevention model

The current study utilized mixed-methods analyses to examine the process of adapting a home-based parenting program for a local Latino community. The study examined the: (a) acceptability and cultural congruence of the adapted SafeCare® protocol, (b) adherence to the core components of SafeCare® while adapting to local community culture, and (c) social validity of the new model in addressing SafeCare® target areas (parenting, home safety, and child health). Participants were 28 Latino mothers and eight providers. After training in the adapted model, providers demonstrated improved knowledge and skills. All providers reached national certification standards for SafeCare®, demonstrating fidelity to the core components of the original model. Positive consumer-provider relationships were developed as reflected by the results on the Working Alliance (collaboration between caregivers and parents). Themes from the integrated results of the social validity measures and individual interviews with parents were perceived benefits of the program on targeted areas and cultural congruency of the approach. Recommendations are to consider using adaptation guidelines as outlined to promote local culturally congruent practices.