Gracelyn Cruden

Overview of Meta-Analyses of the Prevention of Mental Health,Substance Use and Conduct Problems

This review presents findings from an overview of meta-analyses of the effects of prevention and promotion programs to prevent mental health, substance use, and conduct problems. The review of 48 meta-analyses found small but significant changes that reduce depression, anxiety, antisocial behavior, and substance use. Furthermore, the results were sustained over time. Meta-analyses often found that the effects were heterogeneous. A conceptual model is proposed to guide the study of moderators of program effects in future meta-analyses, and methodological issues in synthesizing findings across preventive interventions are discussed.

Advancing Science Through Collaborative Data Sharing and Synthesis

The demand for researchers to share their data has increased dramatically in recent years. There is a need to replicate and confirm scientific findings to bolster confidence in many research areas. Data sharing also serves the critical function of allowing synthesis of findings across trials. As innovative statistical methods have helped resolve barriers to synthesis analyses, data sharing and synthesis can help answer research questions that cannot be answered by individual trials alone. However, the sharing of data among researchers remains challenging and infrequent. This article aims to (a) increase support for data sharing and synthesis collaborations among researchers to advance scientific knowledge and (b) provide a model for establishing these collaborations using the example of the ongoing National Institute of Mental Health’s Collaborative Data Synthesis on Adolescent Depression Trials. This study brings together datasets from existing prevention and treatment trials in adolescent depression, as well as researchers and stakeholders, to answer questions about “for whom interventions work” and “by what pathways interventions have their effects.” This is critical to improving interventions, including increasing knowledge about intervention efficacy among minority populations, or what we call “scientific equity.” The collaborative model described is relevant to fields with research questions that can only be addressed by synthesizing individual-level data.

An Overview of Research and Evaluation Designs for Dissemination and Implementation

The wide variety of dissemination and implementation designs now being used to evaluate and improve health systems and outcomes warrants review of the scope, features, and limitations of these designs. This article is one product of a design workgroup that was formed in 2013 by the National Institutes of Health to address dissemination and implementation research, and whose members represented diverse methodologic backgrounds, content focus areas, and health sectors. These experts integrated their collective knowledge on dissemination and implementation designs with searches of published evaluations strategies. This article emphasizes randomized and nonrandomized designs for the traditional translational research continuum or pipeline, which builds on existing efficacy and effectiveness trials to examine how one or more evidence-based clinical/prevention interventions are adopted, scaled up, and sustained in community or service delivery systems. We also mention other designs, including hybrid designs that combine effectiveness and implementation research, quality improvement designs for local knowledge, and designs that use simulation modeling.

Evaluation of two implementation strategies in 51 child county public service systems in two states: results of a cluster randomized head-to-head implementation trial

BackgroundMuch is to be learned about what implementation strategies are the most beneficial to communities attempting to adopt evidence-based practices. This paper presents outcomes from a randomized implementation trial of Multidimensional Treatment Foster Care (MTFC) in child public service systems in California and Ohio, including child welfare, juvenile justice, and mental health.MethodsFifty-one counties were assigned randomly to one of two different implementation strategies (Community Development Teams (CDT) or independent county implementation strategy (IND)) across four cohorts after being matched on county characteristics. We compared these two strategies on implementation process, quality, and milestone achievements using the Stages of Implementation Completion (SIC) (Implement Sci 6(1):1-8, 2011).ResultsA composite score for each county, combining the final implementation stage attained, the number of families served, and quality of implementation, was used as the primary outcome. No significant difference between CDT and IND was found for the composite measure. Additional analyses showed that there was no evidence that CDT increased the proportion of counties that started-up programs (i.e., placed at least one family in MTFC). For counties that did implement MTFC, those in the CDT condition served over twice as many youth during the study period as did IND. Of the counties that successfully achieved program start-up, those in the CDT condition completed the implementation process more thoroughly, as measured by the SIC. We found no significant differences by implementation condition on the time it took for first placement, achieving competency, or number of stages completed.ConclusionsThis trial did not lead to higher rates of implementation or faster implementation but did provide evidence for more robust implementation in the CDT condition compared to IND implementation once the first family received MTFC services. This trial was successful from a design perspective in that no counties dropped out, even though this study took place during an economic recession. We believe that this methodologic approach of measurement utilizing the SIC, which is comprised of the three dimensions of quality, quantity, and timing, is appropriate for a wide range of implementation and translational studies.Trial registrationTrial ID: NCT00880126 (ClinicalTrials.gov).

Toward Scientific Equity for the Prevention of Depression and Depressive Symptoms in Vulnerable Youth

Certain subgroups of youth are at high risk for depression and elevated depressive symptoms, and experience limited access to quality mental health care. Examples are socioeconomically disadvantaged, racial/ ethnic minority, and sexual minority youth. Research shows that there are efficacious interventions to prevent youth depression and depressive symptoms. These preventive interventions have the potential to play a key role in addressing these mental health disparities by reducing youth risk factors and enhancing protective factors. However, there are comparatively few preventive interventions directed specifically to these vulnerable subgroups, and sample sizes of diverse subgroups in general prevention trials are often too low to assess whether preventive interventions work equally well for vulnerable youth compared to other youth. In this paper, we describe the importance and need for “scientific equity,” or equality and fairness in the amount of scientific knowledge produced to understand the potential solutions to such health disparities. We highlight possible strategies for promoting scientific equity, including the following: increasing the number of prevention research participants from vulnerable subgroups, conducting more data synthesis analyses and implementation science research, disseminating preventive interventions that are efficacious for vulnerable youth, and increasing the diversity of the prevention science research workforce. These strategies can increase the availability of research evidence to determine the degree to which preventive interventions can help address mental health disparities. Although this paper utilizes the prevention of youth depression as an illustrative case example, the concepts are applicable to other health outcomes for which there are disparities, such as substance use and obesity.

Blending Qualitative and Computational Linguistics Methods for Fidelity Assessment: Experience with the Familias Unidas Preventive Intervention.

AbstractCareful fidelity monitoring and feedback are critical to implementing effective interventions. A wide range of procedures exist to assess fidelity; most are derived from observational assessments (Schoenwald and Garland, Psycholog Assess 25:146–156, 2013). However, these fidelity measures are resource intensive for research teams in efficacy/effectiveness trials, and are often unattainable or unmanageable for the host organization to rate when the program is implemented on a large scale. We present a first step towards automated processing of linguistic patterns in fidelity monitoring of a behavioral intervention using an innovative mixed methods approach to fidelity assessment that uses rule-based, computational linguistics to overcome major resource burdens. Data come from an effectiveness trial of the Familias Unidas intervention, an evidence-based, family-centered preventive intervention found to be efficacious in reducing conduct problems, substance use and HIV sexual risk behaviors among Hispanic youth. This computational approach focuses on “joining,” which measures the quality of the working alliance of the facilitator with the family. Quantitative assessments of reliability are provided. Kappa scores between a human rater and a machine rater for the new method for measuring joining reached 0.83. Early findings suggest that this approach can reduce the high cost of fidelity measurement and the time delay between fidelity assessment and feedback to facilitators; it also has the potential for improving the quality of intervention fidelity ratings.

Limitations in Using Multiple Imputation to Harmonize Individual Participant Data for Meta-Analysis

Individual participant data (IPD) meta-analysis is a meta-analysis in which the individual-level data for each study are obtained and used for synthesis. A common challenge in IPD meta-analysis is when variables of interest are measured differently in different studies. The term harmonization has been coined to describe the procedure of placing variables on the same scale in order to permit pooling of data from a large number of studies. Using data from an IPD meta-analysis of 19 adolescent depression trials, we describe a multiple imputation approach for harmonizing 10 depression measures across the 19 trials by treating those depression measures that were not used in a study as missing data. We then apply diagnostics to address the fit of our imputation model. Even after reducing the scale of our application, we were still unable to produce accurate imputations of the missing values. We describe those features of the data that made it difficult to harmonize the depression measures and provide some guidelines for using multiple imputation for harmonization in IPD meta-analysis.

Automatic classification of communication logs into implementation stages via text analysis.

BackgroundTo improve the quality, quantity, and speed of implementation, careful monitoring of the implementation process is required. However, some health organizations have such limited capacity to collect, organize, and synthesize information relevant to its decision to implement an evidence-based program, the preparation steps necessary for successful program adoption, the fidelity of program delivery, and the sustainment of this program over time. When a large health system implements an evidence-based program across multiple sites, a trained intermediary or broker may provide such monitoring and feedback, but this task is labor intensive and not easily scaled up for large numbers of sites.We present a novel approach to producing an automated system of monitoring implementation stage entrances and exits based on a computational analysis of communication log notes generated by implementation brokers. Potentially discriminating keywords are identified using the definitions of the stages and experts’ coding of a portion of the log notes. A machine learning algorithm produces a decision rule to classify remaining, unclassified log notes.ResultsWe applied this procedure to log notes in the implementation trial of multidimensional treatment foster care in the California 40-county implementation trial (CAL-40) project, using the stages of implementation completion (SIC) measure. We found that a semi-supervised non-negative matrix factorization method accurately identified most stage transitions. Another computational model was built for determining the start and the end of each stage.ConclusionsThis automated system demonstrated feasibility in this proof of concept challenge. We provide suggestions on how such a system can be used to improve the speed, quality, quantity, and sustainment of implementation. The innovative methods presented here are not intended to replace the expertise and judgement of an expert rater already in place. Rather, these can be used when human monitoring and feedback is too expensive to use or maintain. These methods rely on digitized text that already exists or can be collected with minimal to no intrusiveness and can signal when additional attention or remediation is required during implementation. Thus, resources can be allocated according to need rather than universally applied, or worse, not applied at all due to their cost.

Increasing the Delivery of Preventive Health Services in Public Education.

The delivery of prevention services to children and adolescents through traditional healthcare settings is challenging for a variety of reasons. Parent- and community-focused services are typically not reimbursable in traditional medical settings, and personal healthcare services are often designed for acute and chronic medical treatment rather than prevention. To provide preventive services in a setting that reaches the widest population, those interested in public health and prevention often turn to school settings. This paper proposes that an equitable, efficient manner in which to promote health across the life course is to integrate efforts from public health, primary care, and public education through the delivery of preventive healthcare services, in particular, in the education system. Such an integration of systems will require a concerted effort on the part of various stakeholders, as well as a shared vision to promote child health via community and institutional stakeholder partnerships. This paper includes (1) examination of some key system features necessary for delivery of preventive services that improve child outcomes; (2) a review of the features of some common models of school health services for their relevance to prevention services; and (3) policy and implementation strategy recommendations to further the delivery of preventive services in schools. These recommendations include the development of common metrics for health outcomes reporting, facilitated data sharing of these metrics, shared organization incentives for integration, and improved reimbursement and funding opportunities.

A statistical method for synthesizing mediation analyses using the product of coefficient approach across multiple trials

Mediation analysis often requires larger sample sizes than main effect analysis to achieve the same statistical power. Combining results across similar trials may be the only practical option for increasing statistical power for mediation analysis in some situations. In this paper, we propose a method to estimate: (1) marginal means for mediation path a, the relation of the independent variable to the mediator; (2) marginal means for path b, the relation of the mediator to the outcome, across multiple trials; and (3) the between-trial level variance–covariance matrix based on a bivariate normal distribution. We present the statistical theory and an R computer program to combine regression coefficients from multiple trials to estimate a combined mediated effect and confidence interval under a random effects model. Values of coefficients a and b, along with their standard errors from each trial are the input for the method. This marginal likelihood based approach with Monte Carlo confidence intervals provides more accurate inference than the standard meta-analytic approach. We discuss computational issues, apply the method to two real-data examples and make recommendations for the use of the method in different settings.