Marley W. Watkins

ADHD and Achievement Meta-Analysis of the Child, Adolescent, and Adult Literatures and a Concomitant Study With College Students

This article presents results from two interrelated studies. The first study conducted a meta-analysis of the published literature since 1990 to determine the magnitude of achievement problems associated with attention-deficit/hyperactivity disorder (ADHD). Effect sizes were significantly different between participants with and without ADHD (sample weighted r = .32, sample weighted d = . 71; p = .001). Effects were also examined according to the moderators of age, gender, achievement domain (reading, math, spelling), measurement method (standardized tests vs. grades, parent/teacher ratings, etc.), sample type (clinical vs. nonclinical), and system used to identify ADHD (DSM-III-R vs. DSM-IV). Significant differences emerged from the moderator comparisons. The second study, using averaged effect sizes from the first study as a baseline for comparison, investigated achievement levels for an understudied age group with ADHD, namely, college students. Unlike previous studies at the college level, the sample incorporated both student and parent ratings (N = 380 dyads). The results were comparable to outcomes from the meta-analysis for college students and adults. Analyses demonstrated modest (R = .21) but meaningful predictive validity across 1 year to end-of-first-year grades. However, unlike earlier studies with children and adolescents, student ratings were as predictive as parent ratings. Findings are discussed in terms of the impact of moderator variables on ADHD and achievement.

Teacher ratings of Attention-Deficit/Hyperactivity Disorder symptoms: Factor structure and normative data.

Comprehensive assessment of attention-deficit/hyperactivity disorder (ADHD) symptoms includes parent and teacher questionnaires. The ADHD Rating Scale-5 was developed to incorporate changes for the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association, 2013). This study examined the fit of a correlated, 2-factor structure of ADHD (i.e., DSM-5 conceptual model) and alternative models; determined whether ADHD symptom ratings varied across teacher and child demographic characteristics; and presented normative data. Two samples were included: (a) 2,079 parents and guardians (1,131 female, 948 male) completed ADHD symptom ratings for children (N = 2,079; 1,037 males, 1,042 females) between 5 and 17 years old (M = 10.68; SD = 3.75) and (b) 1,070 teachers (766 female, 304 male) completed ADHD symptom ratings for students (N = 2,140; 1,070 males, 1,070 females) between 5 and 17 years old (M = 11.53; SD = 3.54) who attended kindergarten through 12th grade. The 2-factor structure was confirmed for both parent and teacher ratings and was invariant across child gender, age, informant, informant gender, and language. In general, boys were higher in symptom frequency than girls; older children were rated lower than younger children, especially for hyperactivity-impulsivity; and non-Hispanic children were rated higher than Hispanic children. Teachers also rated non-Hispanic African American children higher than non-Hispanic White, Asian, and Hispanic children. Non-Hispanic White teachers provided lower hyperactivity-impulsivity ratings than non-Hispanic, African American, and Hispanic teachers. Normative data are reported separately for parent and teacher ratings by child gender and age. The merits of using the ADHD Rating Scale-5 in a multimodal assessment protocol are discussed. (PsycINFO Database Record

Interobserver Agreement in Behavioral Research: Importance and Calculation

Behavioral researchers have developed a sophisticated methodology to evaluate behavioral change which is dependent upon accurate measurement of behavior. Direct observation of behavior has traditionally been the mainstay of behavioral measurement. Consequently, researchers must attend to the psychometric properties, such as interobserver agreement, of observational measures to ensure reliable and valid measurement. Of the many indices of interobserver agreement, percentage of agreement is the most popular. Its use persists despite repeated admonitions and empirical evidence indicating that it is not the most psychometrically sound statistic to determine interobserver agreement due to its inability to take chance into account. Cohens (1960) kappa has long been proposed as the more psychometrically sound statistic for assessing interobserver agreement. Kappa is described and computational methods are presented.

Long-Term Stability of the Wechsler Intelligence Scale for Children--Third Edition.

Long-term stability of the Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV; Wechsler, 2003) was investigated with a sample of 344 students from 2 school districts twice evaluated for special education eligibility at an average interval of 2.84 years. Test-retest reliability coefficients for the Verbal Comprehension Index (VCI), Perceptual Reasoning Index (PRI), Working Memory Index (WMI), Processing Speed Index (PSI), and the Full Scale IQ (FSIQ) were .72, .76, .66, .65, and .82, respectively. As predicted, the test-retest reliability coefficients for the subtests (Mdn = .56) were generally lower than the index scores (Mdn = .69) and the FSIQ (.82). On average, subtest scores did not differ by more than 1 point, and index scores did not differ by more than 2 points across the test-retest interval. However, 25% of the students earned FSIQ scores that differed by 10 or more points, and 29%, 39%, 37%, and 44% of the students earned VCI, PRI, WMI, and PSI scores, respectively, that varied by 10 or more points. Given this variability, it cannot be assumed that WISC-IV scores will be consistent across long test-retest intervals for individual students.

ADHD and college students: Exploratory and confirmatory factor structures with student and parent data

Exploratory factor analyses (EFAs) and confirmatory factor analyses (CFAs) were used to investigate the structure of the Student Report Inventory (SRI) and Parent Report Inventory (PRI) of the College Attention-Deficit/Hyperactivity Disorder (ADHD) Response Evaluation. The sample was composed of 1,080 college students and their parents and was stratified by ethnicity, gender, ability level, age, grade, region of residence, and psychoeducational classification status. Results varied according to the information source (self-report vs. parent). EFA uncovered and CFA confirmed 3 distinct and reliable dimensions for student reports: Inattention, Hyperactivity, and Impulsivity. By contrast, EFA and CFA uncovered a reliable 2-dimension structure for the parent-report data. Factor structures replicated across genders (3 factors for the SRI, and 2 factors for the PRI). Results are discussed in terms of the divergence of structures.

Long-Term Stability of Children's Attitudes Toward Reading

The long-term stability of the construct of childrens attitudes toward reading was examined. One hundred eighty-nine students in Grades 1-4 completed two administrations of the Elementary Reading Attitude Survey (ERAS) following a 3-year interval. Initially, reading attitudes were relatively positive and comparable to the standardization sample for both the recreational and academic subscales of the ERAS. Following the 3 years, however, reading attitudes dropped significantly for both recreational and academic scores. Nonsignificant grade differences were observed, but girls consistently expressed more positive attitudes toward recreational reading than did boys. Similarly, girls also demonstrated greater stability in reading attitudes than boys, based on higher correlations between first and second administrations of the ERAS. Implications for the classroom and directions for future research are provided.

Distinctions Without a Difference The Utility of Observed Versus Latent Factors From the WISC-IV in Estimating Reading and Math Achievement on the WIAT-II

This study employed observed factor index scores as well as latent ability constructs from the Wechsler Intelligence Scale for Children—Fourth Edition (WISC-IV; Wechsler, 2003) in estimating reading and mathematics achievement on the Wechsler Individual Achievement Test—Second Edition (WIAT-II; Wechsler, 2002). Participants were the nationally stratified linking sample (N = 498) of the WISC-IV and WIAT-II. Observed scores from the WISC-IV were analyzed using hierarchical multiple regression analysis. Although the factor index scores provided a statistically significant increment over the Full Scale IQ, the size of the improvement was too small to be of clinical utility. Observed WISC-IV subtest scores were also subjected to structural equation modeling (SEM) analyses. Subtest scores from the WISC-IV were fit to a general factor (g) and four ability constructs corresponding to factor indexes from the WISC-IV (Verbal Comprehension, Perceptual Reasoning, Working Memory, and Processing Speed). For both reading and mathematics, only g (.55 and .77, respectively) and Verbal Comprehension (.37 and .17, respectively) were significant influences. Thus, when using observed scores to predict reading and mathematics achievement, it may only be necessary to consider the Full Scale IQ. In contrast, both g and Verbal Comprehension may be required for explanatory research.

Bifactor Modeling and the Estimation of Model-Based Reliability in the WAIS-IV

Previous confirmatory factor analytic research that has examined the factor structure of the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV) has endorsed either higher order models or oblique factor models that tend to amalgamate both general factor and index factor sources of systematic variance. An alternative model that has not yet been examined for the WAIS-IV is the bifactor model. Bifactor models allow all subtests to load onto both the general factor and their respective index factor directly. Bifactor models are also particularly amenable to the estimation of model-based reliabilities for both global composite scores (ω h ) and subscale/index scores (ω s ). Based on the WAIS-IV normative sample correlation matrices, a bifactor model that did not include any index factor cross loadings or correlated residuals was found to be better fitting than the conventional higher order and oblique factor models. Although the ω h estimate associated with the full scale intelligence quotient (FSIQ) scores was respectably high (.86), the ω s estimates associated with the WAIS-IV index scores were very low (.13 to .47). The results are interpreted in the context of the benefits of a bifactor modeling approach. Additionally, in light of the very low levels of unique internal consistency reliabilities associated with the index scores, it is contended that clinical index score interpretations are probably not justifiable.

Illusions of Meaning in the Ipsative Assessment of Children's Ability

In this study, we analyze the relative efficacy of normative and ipsative measures for the study of intra- and interindividual differences in child ability. With the use of representative data sets, including the WISC-R national standardization sample, purely ipsatized (or deviational ipsative) subtest scores were contrasted with conventional norm-based scores in terms of the evidential and consequential bases for validity. Internal and external evidence for validity was assessed for relative convergence of ability attributes, short- and long-term stability, and predictive efficiency. Comparative utility of each type of measure was explored for theoretical relevance, applicability in measurement work, and assessment of individualized intervention outcomes. Ipsative ability measures were found to be uniformly inferior to their normative counterparts, with ipsative measures conveying no uniquely useful information and otherwise impeding the versatility of assessment.

Reading Motivation: Multidimensional and Indeterminate.

The Motivations for Reading Questionnaire (MRQ) contains 54 items that are posited to tap 11 dimensions of reading motivation. The structural validity of the MRQ was investigated with 2 samples: (a) 328 students in Grades 3–5 from 2 suburban mid-Atlantic elementary schools and (b) 735 students in Grades 3–5 in 2 suburban southwestern elementary schools. With confirmatory factor analyses (CFAs), the theoretical 11-factor structure did not adequately fit the data in either sample. Subsequently, exploratory factor analyses found 8 factors for each sample with 6 factors defined by only 3 or 4 common items. However, a double CFA cross-validation found an inadequate fit for both samples. Given these results, the authors suggest that the MRQ be revised.