Thomas R. Kratochwill

Single-Case Intervention Research Design Standards

In an effort to responsibly incorporate evidence based on single-case designs (SCDs) into the What Works Clearinghouse (WWC) evidence base, the WWC assembled a panel of individuals with expertise in quantitative methods and SCD methodology to draft SCD standards. In this article, the panel provides an overview of the SCD standards recommended by the panel (henceforth referred to as the Standards) and adopted in Version 1.0 of the WWC’s official pilot standards. The Standards are sequentially applied to research studies that incorporate SCDs. The design standards focus on the methodological soundness of SCDs, whereby reviewers assign the categories of Meets Standards, Meets Standards With Reservations, and Does Not Meet Standards to each study. Evidence criteria focus on the credibility of the reported evidence, whereby the outcome measures that meet the design standards (with or without reservations) are examined by reviewers trained in visual analysis and categorized as demonstrating Strong Evidence, Moderate Evidence, or No Evidence. An illustration of an actual research application of the Standards is provided. Issues that the panel did not address are presented as priorities for future consideration. Implications for research and the evidence-based practice movement in psychology and education are discussed. The WWC’s Version 1.0 SCD standards are currently being piloted in systematic reviews conducted by the WWC. This document reflects the initial standards recommended by the authors as well as the underlying rationale for those standards. It should be noted that the WWC may revise the Version 1.0 standards based on the results of the pilot; future versions of the WWC standards can be found at http://www.whatworks.ed.gov.

Enhancing the Scientific Credibility of Single-Case Intervention Research: Randomization to the Rescue.

In recent years, single-case designs have increasingly been used to establish an empirical basis for evidence-based interventions and techniques in a variety of disciplines, including psychology and education. Although traditional single-case designs have typically not met the criteria for a randomized controlled trial relative to conventional multiple-participant experimental designs, there are procedures that can be adopted to create a randomized experiment in this class of experimental design. Our two major purposes in writing this article were (a) to review the various types of single-case design that have been and can be used in psychological and educational intervention research and (b) to incorporate randomized experimental schemes into these designs, thereby improving them so that investigators can draw more valid conclusions from their research. For each traditional single-case design type reviewed, we provide illustrations of how various forms of randomization can be introduced into the basic design structure. We conclude by recommending that traditional single-case intervention designs be transformed into more scientifically credible randomized single-case intervention designs whenever the research conditions under consideration permit.

A meta-systems approach to evidence–based practice for children and adolescents

Improving outcomes for children and adolescents with mental health needs demands a broad meta-systemic orientation to overcome persistent problems in current service systems. Improving outcomes necessitates inclusion of current and emerging evidence about effective practices for the diverse population of youth and their families. Key components of the meta-system for children with emotional or behavioral needs include families, cultural norms and values, and service sectors such as schools, pediatric health centers, specialty mental health systems, juvenile justice systems, child protection services, and substance use treatment systems. We describe each component of the meta-system, noting challenges to the provision of evidence-based practice (EBP) and highlighting ways to optimize outcomes. Our focus is on the inclusion of evidence-based assessment and interventions, including prevention, within a developmentally driven and culturally responsive contextual model. Recommendations for addressing disparities in research funding and essential steps to foster communication and coordination of EBP across settings are provided.

Behavioral Parent-Teacher Consultation: Conceptual and Research Considerations

The importance of involving parents in their childrens education has been documented consistently. Likewise, school psychology as a profession traditionally has recognized the importance of working actively and collaboratively with parents. Little conceptual or empirical work has been reported, however, that links home and school individuals systematically in collaborative problem-solving consultation. Behavioral consultation provides a useful framework for working within and between family and school systems to involve parents and teachers together in cooperative problem-solving, with a focus on the interacting systems in a childs life. The potential advantages of having both parents and teachers serve as consultees are that this structure promotes (a) identification of temporally or contextually distal setting events, (b) consistent and systematic behavioral programming, and (c) programming for generalization of treatment effects across settings. Likewise, the potential for effective communication, constructive partnerships, and productive relationships between home and school is increased. In the present article, we present an overview of conjoint behavioral consultation, as well as procedural considerations and future research directions.

Single Subject Designs A Perspective on the Controversy Over Employing Statistical Inference and Implications for Research and Training in Behavior Modification

Issues surrounding the application of inferential statistical tests to behavior modification research was presented. Four major behavior modification journals were surveyed from their inception through 1974 for their use of inferential statistical tests in empirical research studies. Results indicated that Behaviour Research and Therapy and Behavior Therapy contained the highest percentage of articles using inferential statistical procedures. Over all journals, nearly all studies employing group designs used inferential statistical tests. Parametric analysis-of-variance and various nonparametric tests were generally the most commonly employed statistics within the journals sampled. Implications of the review suggest that the methodological base of behavior modification research is becoming more diverse as reflected in increased use of group designs where inferential tests are commonly employed. While all journals contained some N=1 experiments which relied on some form of inferential statistics, both valid and invalid applications are noted. Issues surrounding the use of statistics are documented. Implications for research and training are discussed.

Preparing Psychologists for Evidence-Based School Practice: Lessons Learned and Challenges Ahead.

The evidence-based practice (EBP) movement has the potential to significantly advance the quality of psychological and educational services provided by psychologists working in schools. Training psychologists in EBP has challenged the profession and caused faculty in graduate programs to reevaluate and retool professional training curricula and instructional practices. Four domains of challenges in graduate training are identified: (a) integrating the EBP knowledge base into the curriculum, (b) expanding models of research training, (c) expanding training in prevention science, and (d) expanding training in problem-solving consultation and school contextual issues. For each of these, the author discusses the range and scope of the challenge and possible solutions for advancing graduate training in psychology relevant to school practice.

Meta-analysis for single-case consultation outcomes: Applications to research and practice

Abstract The application of meta-analysis holds much appeal for single-case consultation outcome research. We review a meta-analytic method for using within-study treatment effect sizes in reporting consultation outcomes. The strengths and limitations of traditional group design meta-analysis are examined. Various methods for analyzing single-case outcomes are discussed briefly, followed by an examination of the use of meta-analysis in single-case reviews across independent studies. Within-study meta-analytic results are presented that were derived from treatments implemented in consultations in natural settings. To conclude the article, an illustration is offered of a single-case data analysis display that incorporates meta-analytic results along with other indices of treatment outcome. Recommendations are provided for using meta-analytic methods to evaluate outcomes of single-case consultation treatment.

Reduction of frequent stomachache complaints by time out

Time out was employed to treat a 10-year-old girls report of stomach pain complaints in the home and at school. Time out involved removing the girl from adult attention and activities for verbal pain complaint episodes. Pain complaint episodes were treated sequentially in a multiple-baseline design across home and school environments. The treatment program rapidly reduced pain episodes over both settings. Follow-ups at 4, 6, and 8 weeks after the program terminated indicated that episodes remained at zero levels. A 1-year follow-up also indicated that there were no further problems with stomach pain episodes.

Single-case intervention research : methodological and statistical advances

Thanks to remarkable methodological and statistical advances in recent years, Single-Case design (SCD) research has become a viable and often essential option for researchers in applied psychology, education, and related fields. This text is a compendium of information and tools for researchers considering SCD research, a methodology in which one or several participants (or other units) comprise a systematically-controlled experimental intervention study. SCD is a highly flexible method of conducting applied intervention research where it is not feasible or practical to collect data from traditional groups of participants. Initial chapters lay out the key components of SCDs, from articulating dependent variables to documenting methods for achieving experimental control and selecting an appropriate design model. Subsequent chapters show when and how to implement SCDs in a variety of contexts and how to analyze and interpret results. Authors emphasize key design and analysis tactics, such as randomization, to help enhance the internal validity and scientific credibility of individual studies. This rich resource also includes in-depth descriptions of large-scale SCD research projects being undertaken at key institutions; practical suggestions from journal editors on how to get SCD research published; and detailed instructions for free, user-friendly, web-based randomization software.

Response to Intervention: Conceptual and Methodological Issues in Implementation

Thomas R. Kratochwill, PhD, is Sears–Bascom Professor, School Psychology Program, University of Wisconsin–Madison. tomkat@education.wisc.edu Melissa A. Clements, PhD, is a Research Scientist at the Wisconsin Center for Educational Research, University of Wisconsin–Madison. maclements@wisc.edu Kristen M. Kalymon, MS, is a graduate student in the School Psychology Program, University of Wisconsin–Madison. kmkalymon@wisc.edu