Ryan J. McGill

Exploratory and Hierarchical Factor Analysis of the WJ-IV Cognitive at School Age.

Exploratory and confirmatory factor analytic studies were not reported in the Technical Manual for the Woodcock-Johnson, 4th ed. Cognitive (WJ IV Cognitive; Schrank, McGrew, & Mather, 2014b) Instead, the internal structure of the WJ IV Cognitive was extrapolated from analyses based on the full WJ IV test battery (Schrank, McGrew, & Mather, 2014b). Even if the veracity of extrapolating from the WJ IV full battery were accepted, there were shortcomings in the choices of analyses used and only limited information regarding those analyses was presented in the WJ IV Technical Manual (McGrew, Laforte, & Shrank, 2014). The present study examined the structure of the WJ IV Cognitive using exploratory factor analysis procedures (principal axis factoring with oblique [promax] rotation followed by application of the Schmid–Leiman, 1957, procedure) applied to standardization sample correlation matrices for 2 school age groups (ages 9–13; 14–19). Four factors emerged for both the 9–13 and 14–19 age groups in contrast to the publisher’s proposed 7 factors. Results of these analyses indicated a robust manifestation of general intelligence (g) that exceeded the variance attributed to the lower-order factors. Model-based reliability estimates supported interpretation of the higher-order factor (i.e., g). Additional analyses were conducted by forcing extraction of the 7 theoretically posited factors; however, the resulting solution was only partially aligned (i.e., Gs, Gwm) with the theoretical structure promoted in the Technical Manual and suggested the preeminence of the higher-order factor. Results challenge the hypothesized structure of the WJ IV Cognitive and raise concerns about its alignment with Cattell-Horn-Carroll theory.

Incremental Validity of the WJ III COG: Limited Predictive Effects Beyond the GIA-E

This study is an examination of the incremental validity of Cattell-Horn-Carroll (CHC) broad clusters from the Woodcock-Johnson III Tests of Cognitive Abilities (WJ III COG) for predicting scores on the Woodcock-Johnson III Tests of Achievement (WJ III ACH). The participants were children and adolescents, ages 6-18 (n = 4,722), drawn from the WJ III standardization sample. The sample was nationally stratified and proportional to U.S. census estimates for race/ethnicity, parent education level, and geographic region. Hierarchical multiple regression analyses were used to assess for cluster-level effects after controlling for the variance accounted for by the General Intellectual Ability-Extended (GIA-E) composite score. The results were interpreted using the R²/ΔR² statistic as the effect size indicator. Consistent with previous studies, the GIA-E accounted for statistically and clinically significant portions of WJ III ACH cluster score variance, with R2 values ranging from .29 to .56. WJ III COG CHC cluster scores collectively provided statistically significant incremental variance beyond the GIA-E in all of the regression models, although the effect sizes were consistently negligible to small (Average ΔR2(CHC) = .06), with significant effects observed only in the Oral Expression model (ΔR²(CHC) = .23). Individually, the WJ III COG cluster scores accounted for mostly small portions of achievement variance across the prediction models, with a large effect found for the Comprehension-Knowledge cluster in the Oral Expression model (ΔR²(Gc) = .23). The potential clinical and theoretical implications of these results are discussed.

Critical Issues in Specific Learning Disability Identification What We Need to Know About the PSW Model

As a result of the upcoming Federal reauthorization of the Individuals With Disabilities Education Improvement Act (IDEA), practitioners and researchers have begun vigorously debating what constitutes evidence-based assessment for the identification of specific learning disability (SLD). This debate has resulted in strong support for a method that appraises an individual’s profile of cognitive test scores for the purposes of determining cognitive processing strengths and weaknesses, commonly referred to as patterns of strengths and weaknesses (PSW). Following the Fuchs and Deshler model, questions regarding the psychometric and conceptual integrity of the PSW model are addressed. Despite the strong claims made by many PSW proponents, the findings by this review demonstrate the need for additional information to determine whether PSW is a viable alternative to existing eligibility models and worthy for large scale adoption for SLD identification. Implications for public policy and future SLD research are also discussed.

Orthogonal Higher Order Structure of the WISC-IV Spanish Using Hierarchical Exploratory Factor Analytic Procedures:

As recommended by Carroll, the present study examined the factor structure of the Wechsler Intelligence Scale for Children–Fourth Edition Spanish (WISC-IV Spanish) normative sample using higher order exploratory factor analytic techniques not included in the WISC-IV Spanish Technical Manual. Results indicated that the WISC-IV Spanish subtests were properly aligned with theoretically proposed factors; however, application of the Schmid and Leiman procedure found that the g factor accounted for large portions of total and common variance, whereas the four first-order factors accounted for small portions of total and common variance. Implications for clinical interpretation of the measurement instrument are discussed.

Exploratory Higher Order Analysis of the Luria Interpretive Model on the Kaufman Assessment Battery for Children-Second Edition (KABC-II) School-Age Battery.

Higher order factor structure of the Luria interpretive scheme on the Kaufman Assessment Battery for Children–Second Edition (KABC-II) for the 7- to 12-year and the 13- to 18-year age groups in the KABC-II normative sample (N = 2,025) is reported. Using exploratory factor analysis, multiple factor extraction criteria, and hierarchical exploratory factor analysis not included in the KABC-II manual, two-, three-, and four-factor extractions were analyzed to assess the hierarchical factor structure by sequentially partitioning variance appropriately to higher order and lower order dimensions as recommended by Carroll. No evidence for a four-factor solution was found. Results showed that the largest portions of total and common variance were accounted for by the second-order general factor and that interpretation should focus primarily, if not exclusively, at that level of measurement.

Incremental Criterion Validity of the WJ-III COG Clinical Clusters: Marginal Predictive Effects beyond the General Factor.

The current study examined the incremental validity of the clinical clusters from the Woodcock–Johnson III Tests of Cognitive Abilities (WJ-III COG) for predicting scores on the Woodcock–Johnson III Tests of Achievement (WJ-III ACH). All participants were children and adolescents (N = 4,722) drawn from the nationally representative WJ-III standardization sample. Hierarchical multiple regression analyses were used to assess for cluster-level effects after controlling for the variance accounted for by the General Intellectual Ability (GIA) composite score. Consistent with previous studies, the GIA accounted for clinically significant portions of WJ-III ACH score variance in all of the regression models with R2 values ranging from .33 to .63. The clinical cluster scores collectively accounted for small to moderate incremental effects with no meaningful effects observed for individual indicators. Potential implications of these results for empirically supported interpretation of the WJ-III COG are discussed.

Hierarchical Exploratory Factor Analyses of the Woodcock-Johnson IV Full Test Battery: Implications for CHC Application in School Psychology.

The Woodcock-Johnson (fourth edition; WJ IV; Schrank, McGrew, & Mather, 2014a) was recently redeveloped and retains its linkage to Cattell-Horn-Carroll theory (CHC). Independent reviews (e.g., Canivez, 2017) and investigations (Dombrowski, McGill, & Canivez, 2017) of the structure of the WJ IV full test battery and WJ IV Cognitive have suggested the need for additional factor analytic exploration. Accordingly, the present study used principal axis factoring (PAF) followed by the Schmid and Leiman (SL; Schmid & Leiman, 1957) procedure with the 2 school-aged correlation matrices from the normative sample to determine the degree to which the WJ IV total battery structure could be replicated. Although 7 factors emerged across the 9 to 19 age range, the pattern of subtests loadings did not fully cohere with the structure presented in the Technical Manual, most notably for the academic fluency subtests. Also, the Fluid Reasoning (Gf) and Quantitative Reasoning (Gq) subtests coalesced to form a combined factor rather than 2 separate factors and the Long Term Retrieval (Gltr) subtests aligned with a variety of different factors. The results of this study indicated that the general intelligence factor variance far exceeded the variance attributed to the lower-order CHC factors. The combination of subtest migration and nominal total/common variance of the CHC lower-order factors suggests caution when interpreting the myriad CHC-related indices when making high stakes decisions. Implications for clinical practice are discussed.

Interpretation of KABC-II Scores: An Evaluation of the Incremental Validity of Cattell-Horn-Carroll (CHC) Factor Scores in Predicting Achievement

This study is an examination of the incremental validity of Cattell-Horn-Carroll (CHC) factor scores from the Kaufman Assessment Battery for Children-second edition (KABC-II) for predicting scores on the Kaufman Test of Educational Achievement-second edition (KTEA-II). The participants were children and adolescents, ages 7-18, (N = 2,025) drawn from the KABC-II standardization sample. The sample was nationally stratified and proportional to U.S. census estimates for sex, ethnicity, geographic region, and parent education level. Hierarchical multiple regression analyses were used to assess for factor-level effects after controlling for the variance accounted for by the full scale Fluid-Crystallized Index (FCI) score. The results were interpreted using the R2/ΔR2 statistic as effect size indices. Consistent with similar incremental validity studies, the FCI accounted for statistically and clinically significant portions of KTEA-II score variance, with R2 values ranging from .30 to .65. KABC-II CHC factor scores collectively provided statistically significant incremental variance beyond the FCI in all of the regression models, although the effect size estimates were consistently negligible to small (Average ÄRCHC2 = .03). Individually, the KABC-II factor scores accounted for mostly small portions of achievement variance across the prediction models, with none of the individual CHC factors accounting for clinically significant incremental prediction beyond the FCI. Additionally, most of the unique first-order predictive variance was captured by the Crystallized Ability factor alone. The potential clinical and theoretical implications of these results are discussed.

Social Story™ interventions for decreasing challenging behaviours: a single-case meta-analysis 1995–2012

A meta-analysis of the single-case research examining the efficacy of Social Story™ interventions for decreasing problem behaviours of children and adolescents with autism spectrum disorder was conducted by examining 27 outcome studies (n = 77) between 1995 and 2012 that yielded 64 intervention effects across three single-case outcome indicators. The overall mean visual analysis ratings and percentage of non-overlapping data scores indicated that the use of Social Story™ interventions resulted in small to negligible effects whereas the weighted effect size estimator ( = 0.79) indicated moderate to large treatment effects. Moderator analysis indicated that intervention setting, intervention agent, length of treatment, and publication status were all associated with positive effects for behavioural outcomes, although the significance of these outcomes were not consistent across indicators. Implications for practitioners and clinicians, suggestions for future research, and limitations are discussed.

Investigation of the Factor Structure of the Comprehensive Test of Nonverbal Intelligence–Second Edition (CTONI-2) Using Exploratory Factor Analysis

The present study examined the structure of the Comprehensive Test of Nonverbal Intelligence–Second Edition (CTONI-2) normative sample using exploratory factor analysis, multiple factor extraction criteria, and higher-order exploratory factor analytic techniques that were not reported in the in the CTONI-2 Examiner’s Manual. Results indicated that retention of the hypothesized CTONI-2 measurement model resulted in several theoretically inconsistent factor loadings, low factor loadings, and the retention of impermissible factors (e.g., factors with no salient loadings). Hierarchical exploratory analyses with the Schmid and Leiman procedure found that the second-order g factor accounted for large portions of total and common variance, in contrast to the two first-order factors that accounted for small portions of dimensional variance. Results suggest that the CTONI-2 provides a strong measure of general intelligence and clinical interpretation should be limited to that level of the measurement instrument.