Michael D. Toland

Goal Attainment Scaling as an Outcome Measure in Randomized Controlled Trials of Psychosocial Interventions in Autism.

Goal attainment scaling (GAS) holds promise as an idiographic approach for measuring outcomes of psychosocial interventions in community settings. GAS has been criticized for untested assumptions of scaling level (i.e., interval or ordinal), inter-individual equivalence and comparability, and reliability of coding across different behavioral observation methods. We tested assumptions of equality between GAS descriptions for outcome measurement in a randomized trial (i.e., measurability, equidistance, level of difficulty, comparability of behavior samples collected from teachers vs. researchers and live vs. videotape). Results suggest GAS descriptions can be evaluated for equivalency, that teacher collected behavior samples are representative, and that varied sources of behavior samples can be reliably coded. GAS is a promising measurement approach. Recommendations are provided to ensure methodological quality.

A Randomized Controlled Trial of Compass Web-Based and Face-to-Face Teacher Coaching in Autism.

OBJECTIVE Most children with autism rely on schools as their primary source of intervention, yet research has suggested that teachers rarely use evidence-based practices. To address the need for improved educational outcomes, a previously tested consultation intervention called the Collaborative Model for Promoting Competence and Success (COMPASS; Ruble, Dalrymple, & McGrew, 2010; Ruble, Dalrymple, & McGrew, 2012) was evaluated in a 2nd randomized controlled trial, with the addition of a web-based group. METHOD Forty-nine teacher-child dyads were randomized into 1 of 3 groups: (1) a placebo control (PBO) group, (2) COMPASS followed by face-to-face (FF) coaching sessions, and (3) COMPASS followed by web-based (WEB) coaching sessions. Three individualized goals (social, communication, and independence skills) were selected for intervention for each child. The primary outcome of independent ratings of child goal attainment and several process measures (e.g., consultant and teacher fidelity) were evaluated. RESULTS Using an intent-to-treat approach, findings replicated earlier results with a very large effect size (d = 1.41) for the FF group and a large effect size (d = 1.12) for the WEB group relative to the PBO group. There were no differences in overall change across goal domains between the FF and WEB groups, suggesting the efficacy of videoconferencing technology. CONCLUSIONS COMPASS is effective and results in improved educational outcomes for young children with autism. Videoconferencing technology, as a scalable tool, has promise for facilitating access to autism specialists and bridging the research-to-practice gap.

Practical Guide to Conducting an Item Response Theory Analysis

Item response theory (IRT) is a psychometric technique used in the development, evaluation, improvement, and scoring of multi-item scales. This pedagogical article provides the necessary information needed to understand how to conduct, interpret, and report results from two commonly used ordered polytomous IRT models (Samejima’s graded response [GR] model and reduced GR model). Throughout this article, simulated data from a multi-item scale is used to illustrate IRT analyses. The simulated data and IRTPRO version 2.1 point-and-click commands needed to reproduce all analyses in this article are available as supplemental online materials at http://jea.sagepub.com/maint. The intent of this article is to provide an overview of essential components of an IRT analysis to enable increased access to this powerful tool for applied early adolescence researchers.

Replicating and extending the good-enough level model of change: Considering session frequency

Abstract The good-enough level (GEL) model posits that the rate of change in psychotherapy is related to the total dose of therapy. The psychotherapy dose-response literature has typically measured dose as number of sessions attended without considering the number of days or weeks it takes to complete the sessions (session frequency). The current study sought to replicate the GEL model and explore if session frequency moderates the influence that the number of sessions has on the rate of change in psychotherapy. An archived naturalistic data set with a US university counseling center sample (n=1,207), with treatment progress measured using the Outcome Questionnaire-45 (Lambert et al., 1996), was used. Our results are consistent with the GEL model (i.e., clients who attended fewer sessions evidenced faster rates of change), but extended it by showing that the rate of change was also influenced by session frequency (i.e., clients who attended more sessions on average per week demonstrated more rapid improvement). Evidence suggests that clinicians and researchers should give consideration to session frequency, both in their work with clients and how “dose” is operationalized in psychotherapy research.

Overt and Relational Victimization in Latinos and European Americans: Measurement Equivalence Across Ethnicity, Gender, and Grade Level in Early Adolescent Groups

This study examined the factorial invariance and construct validity equivalence of a self-report of victimization and exclusion (SVEX) for Latino and European American early adolescent participants (fifth and sixth grades; mean age 11.3).The instrument included an expanded set of relational victimization items that more thoroughly tapped exclusion behaviors relevant to developmental and cross-cultural use. Confirmatory factor analyses techniques demonstrated acceptable (partial) factorial invariance across ethnic groups, fifth and sixth graders, and across gender. Linkages between the SVEX scores, peer nominations, internalizing indices, and three demographic variables also supported construct validity equivalence for the SVEX. Findings supported a two-factor model similar to that of Crick and colleagues (e.g., Crick & Grotpeter, 1995) and suggested that the instrument provided an acceptable level of equivalence for overt and relational victimization forms across these groups.

Assessing Mathematics Self-Efficacy: How Many Categories Do We Really Need?.

The present study tested whether a reduced number of categories is optimal for assessing mathematics self-efficacy among middle school students using a 6-point Likert-type format or a 0- to 100-point format. Two independent samples of middle school adolescents (N = 1,913) were administered a 24-item Middle School Mathematics Self-Efficacy Scale using either a 101-point or a 6-point response scale format. The findings suggest that the two different response scale formats were optimally represented by a 4-point scale and supported when samples were pooled. Results provide tentative evidence that middle school students make use of only 4 scale points and that the items on this scale are best matched with adolescents with average to below-average mathematics self-efficacy. Implications for the measurement of self-efficacy and related motivation constructs are discussed, and replications with a 4-point scale using category descriptors for each scale point are needed.

The Interactive Role of Socioeconomic Status, Race/Ethnicity, and Birth Weight on Trajectories of Body Mass Index Growth in Children and Adolescents.

This study assessed how socioeconomic status (SES), race/ethnicity, and birth weight interacted to predict differential patterns of body mass index (BMI) growth among U.S. children born in the early 1990s. Three BMI growth trajectories emerged—one above the 50th percentile across the age range of 5 to 14, one in which children rapidly became obese before adolescence, and one where children started out and remained seriously obese. Hispanic and African American children were more likely to show accelerated patterns of weight gain as were those of lower SES and/or higher birth weights. However, SES interacted with both race/ethnicity and birth weight. For girls of all race/ethnicity groups tested, lower SES and higher birth weights predicted membership in the seriously obese BMI growth trajectory. For African American and Asian boys, however, the higher the SES the more likely they were to be on a trajectory for rapidly developing obesity by early adolescence.

Analysis of Parent, Teacher, and Consultant Speech Exchanges and Educational Outcomes of Students With Autism During COMPASS Consultation

The significant increase in the numbers of students with autism combined with the need for better trained teachers (National Research Council, 2001) call for research on the effectiveness of alternative methods, such as consultation, that have the potential to improve service delivery. Data from 2 randomized controlled single-blind trials indicate that an autism-specific consultation planning framework known as the collaborative model for promoting competence and success (COMPASS) is effective in increasing child Individual Education Programs (IEP) outcomes (Ruble, Dalrymple, & McGrew, 2010; Ruble, McGrew, & Toland, 2011). In this study, we describe the verbal interactions, defined as speech acts and speech act exchanges that take place during COMPASS consultation, and examine the associations between speech exchanges and child outcomes. We applied the Psychosocial Processes Coding Scheme (Leaper, 1991) to code speech acts. Speech act exchanges were overwhelmingly affiliative, failed to show statistically significant relationships with child IEP outcomes and teacher adherence, but did correlate positively with IEP quality.

Introduction to bifactor polytomous item response theory analysis

A bifactor item response theory model can be used to aid in the interpretation of the dimensionality of a multifaceted questionnaire that assumes continuous latent variables underlying the propensity to respond to items. This model can be used to describe the locations of people on a general continuous latent variable as well as on continuous orthogonal specific traits that characterize responses to groups of items. The bifactor graded response (bifac-GR) model is presented in contrast to a correlated traits (or multidimensional GR model) and unidimensional GR model. Bifac-GR model specification, assumptions, estimation, and interpretation are demonstrated with a reanalysis of data (Campbell, 2008) on the Shared Activities Questionnaire. We also show the importance of marginalizing the slopes for interpretation purposes and we extend the concept to the interpretation of the information function. To go along with the illustrative example analyses, we have made available supplementary files that include command file (syntax) examples and outputs from flexMIRT, IRTPRO, R, Mplus, and STATA. Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.jsp.2016.11.001. Data needed to reproduce analyses in this article are available as supplemental materials (online only) in the Appendix of this article.

Mechanisms of Change in COMPASS Consultation for Students with Autism.

More than a decade has passed since the National Research Council described the common elements of effective educational programs for young children with autism. Since that time, few studies have attempted to understand the mechanisms of change and factors affecting the effectiveness of research supported interventions implemented in community settings. Using Dunst’s (2013) framework of implementation science, we examined the relationships between the fidelity of an implementation practice (i.e., a parent–teacher consultation called the Collaborative Model for Promoting Competence and Success; COMPASS), the fidelity of the intervention practice (i.e., teachers’ adherence to the intervention plans generated as a result of COMPASS), and child goal attainment outcomes using data from a randomized controlled trial. Results confirmed the predicted relationships between implementation fidelity, intervention practice fidelity, and child outcomes. Specifically, we replicated findings that two hypothesized mechanisms of change, individual education program (IEP) quality and teacher adherence, positively affected intervention practices directly and child outcomes indirectly.