Kimberly J. Vannest

An improved effect size for single-case research: nonoverlap of all pairs.

Nonoverlap of All Pairs (NAP), an index of data overlap between phases in single-case research, is demonstrated and field tested with 200 published AB contrasts. NAP is a novel application of an established effect size known in various forms as Area Under the Curve (AUC), the Common Language Effect Size (CL), the Probability of Superiority (PS), the Dominance Statistic (DS), Mann-Whitneys U, and Sommers D, among others. NAP was compared with 3 other non-overlap-based indices: PND (percent of nonoverlapping data), PEM (percent of data points exceeding the median), and PAND (percent of all nonoverlapping data), as well as Pearsons R(2). Five questions were addressed about NAP: (a) typical NAP values, (b) its ability to discriminate among typical single-case research results, (c) its power and precision (confidence interval width), (d) its correlation with the established effect size index, R(2), and (e) its relationship with visual judgments. Results were positive, the new index equaling or outperforming the other overlap indices on most criteria.

Combining Nonoverlap and Trend for Single-Case Research: Tau-U

A new index for analysis of single-case research data was proposed, Tau-U, which combines nonoverlap between phases with trend from within the intervention phase. In addition, it provides the option of controlling undesirable Phase A trend. The derivation of Tau-U from Kendalls Rank Correlation and the Mann-Whitney U test between groups is demonstrated. The equivalence of trend and nonoverlap is also shown, with supportive citations from field leaders. Tau-U calculations are demonstrated for simple AB and ABA designs. Tau-U is then field tested on a sample of 382 published data series. Controlling undesirable Phase A trend caused only a modest change from nonoverlap. The inclusion of Phase B trend yielded more modest results than simple nonoverlap. The Tau-U score distribution did not show the artificial ceiling shown by all other nonoverlap techniques. It performed reasonably well with autocorrelated data. Tau-U shows promise for single-case applications, but further study is desirable.

Effect Size in Single-Case Research: A Review of Nine Nonoverlap Techniques

With rapid advances in the analysis of data from single-case research designs, the various behavior-change indices, that is, effect sizes, can be confusing. To reduce this confusion, nine effect-size indices are described and compared. Each of these indices examines data nonoverlap between phases. Similarities and differences, both conceptual and computational, are highlighted. Seven of the nine indices are applied to a sample of 200 published time series data sets, to examine their distributions. A generic meta-analytic method is presented for combining nonoverlap indices across multiple data series within complex designs.

Percentage of All Non-Overlapping Data (PAND) An Alternative to PND

Although single-case researchers are not accustomed to analyzing data statistically, standards for research and accountability from government and other funding agents are creating pressure for more objective, reliable data. In addition, “evidence-based interventions” movements in special education, clinical psychology, and school psychology imply reliable data summaries. Within special education, two heavily debated single-case research (SCR) statistical indices are “percentage of non-overlapping data” (PND) and the regression effect size, R2 . This article proposes a new index—PAND, the “percentage of all non-overlapping data”—to remedy deficiencies of both PND and R2 . PAND is closely related to the established effect size, Pearsons Phi , the “fourfold point correlation coefficient.” The PAND/ Phi procedure is demonstrated and applied to 75 published multiple baseline designs to answer questions about typical effect sizes, relationships with PND and R2 , statistical power, and time efficiency. Confidence intervals and p values for Phi also are demonstrated. The findings are that PAND/ Phi and PND correlate equally well to R2 . However, only PAND/Phi could show adequate power for most of the multiple baseline designs sampled. The findings suggest that PAND/Phi may meet the requirement for a useful effect size for multiple baseline and other longer designs in SCR.

The Improvement Rate Difference for Single-Case Research

This article describes and field-tests the improvement rate difference (IRD), a new effect size for summarizing single-case research data. Termed “risk difference” in medical research, IRD expresses the difference in successful performance between baseline and intervention phases. IRD can be calculated from visual analysis of nonoverlapping data, and is easily explained to most educators. IRD entails few data assumptions and has confidence intervals. The article applies IRD to 166 published data series, correlates results with three other effect sizes: R2, Kruskal-Wallis W, and percent of nonoverlapping data (PND), and reports interrater reliability of the IRD hand scoring. The major finding is that IRD is a promising effect size for single-case research.

Teacher Time Use in Special Education

How special education teachers spend their time is largely unknown. Yet conceptually, “time” is one of the most tangible and salient variables of the effective instruction literature, Carrolls model of school learning and many economic models of performance measures. This currently unknown use of teacher time has clear and important implications for special education research and practice that include teacher quality, the professional roles of educators, accountability and student achievement. 36 special education teachers representing 4 variations of instructional arrangements recorded 2200 hours of data in the spring of 2006. Special educators reported their time use via a web-based monitoring system while continuous and interval direct observation data were simultaneously collected. Data provide a snapshot view of teacher time use and reflect the percentages of a school day spent in academic instruction, non-academic instruction, instructional support, consultation/collaboration, assessment, planning/preparation, discipline, supervision, paperwork, and other responsibilities throughout the year.

Praise Counts: Using Self-Monitoring to Increase Effective Teaching Practices.

The authors examined the effectiveness of self-monitoring for increasing the rates of teacher praise statements and the acceptability of using this technique for teachers. The participant was a first-year teacher of high school students with emotional and behavioral disturbances. The authors completed this study in an ABA maintenance design. They gathered data by direct-observation frequency count sampling in 10-minute continuous intervals. Results indicate self-monitoring to have an effect size of 0.8365 as derived from a simple mean test on the rates of nonbehavior specific praise statements, an effect size of 0.9023 on the frequency of behavior-specific praise statements and an overall effect of 0.9230. In all, this studys results support the use of self-monitoring to increase effective teaching practices, namely praise, and further demonstrates high social validity for the participant and the students.

Adequate Yearly Progress for Students With Emotional and Behavioral Disorders Through Research-Based Practices

Because schools are held accountable for the academic performance of all students, it is important to focus on academics and the need for effective teaching practices. Adequate yearly progress, a method of accountability that is part of the No Child Left Behind Act (2001), profoundly affects the education of students who have emotional and behavioral disorders (EBD). These students, who typically and consistently perform below grade level, are, or soon will be, tested on grade level across content-area courses. The authors conducted a review of academic interventions for students with EBD to broaden the impact of research by developing a list of instructional interventions that researchers have demonstrated to be effective in improving academic performance of students with EBD.

Incorporating nonoverlap indices with visual analysis for quantifying intervention effectiveness in single-case experimental designs

The field of neuropsychological rehabilitation frequently employs single case experimental designs (SCED) in research, but few if any, of the published studies use the effect sizes recommended by the American Psychological Association. Among the available methods for analysing single case designs, this paper focuses on nonoverlap methods. This paper provides examples and suggestions for integrating visual and statistical analysis, pointing out where contradictions may occur and how to be a critical consumer.

Improvement Rate Differences of Academic Interventions for Students With Emotional and Behavioral Disorders

Academic interventions for students with emotional and behavioral disorders (EBD) is a critical area of practice and one that has not been fully developed by large-scale research. Students with EBD are characterized by an ability to achieve academically but demonstrate a failure to do so. Some research on effective instructional practices for students with EBD does exist, but the predominance of single-case research in this area does not typically demonstrate or report a statistical effect size, making the selection and application of effective instructional strategies sometimes difficult. This article provides a thorough review of the existing research on instructional interventions for students with EBD and calculates effect sizes so that recommending effective instructional practices is more easily accomplished. The authors found 16 “types” of academic interventions and reported the individual and mean improvement rate differences.