Gregg M. Macmann

Early Intervention and the Assessment of Developmental Skills: Challenges and Directions

With the advent of recent legislation (P.L. 99–457, amended by P.L. 101–476), practitioners must apply what is known regarding the amelioration of childrens educational and psychological problems through early intervention. Early intervention holds considerable promise, but the benefits are far from guaranteed. There are many questions about the identification of children, their needs, and the qualities of appropriate services. Although the new legislation has provided a framework for the attainment of early intervention goals, practitioners must deal with fallible assessment techniques, difficult intervention decisions, and unknown outcomes for children. For many reasons, early intervention decisions are not well served by traditional developmental assessments. Following a critique of conceptual and technical problems for developmental assessment, we review promising approaches to educational planning for young children. The basics are founded on ecobehavioral analysis and principles of intervention design.

Decision Reliability and Validity: Contributions and Limitations of Alternative Assessment Strategies

The need to develop alternative models of psycho-educational assessment, more attuned to the logical and empirical requirements of intervention design, has been a central theme of school psychology and special education reform. In response to these challenges, alternative assessment and service delivery models have been proposed in which classification and assessment strategies are linked more directly to principles of intervention design. Yet, the degree to which alternative assessment models may share a number of problems with traditional diagnostic assessment practices has not been analyzed sufficiently. In this paper, the concepts of decision reliability and validity, extensions of reliability and validity theory that encompass decision outcomes, are used to frame a general analysis of three alternative assessment strategies: multiple gating, template matching, and time-series methods. We argue that relatively little knowledge may be gained through the analysis of individual scales and assessment technologies without placing them within a decision-making context and studying individual outcomes.

Technical Adequacy for Response to Intervention Practices

Response-to-intervention (RTI) technical adequacy standards should follow from model purpose, procedural specification, procedural adherence, outcome determination, and subsequent plans. Therefore, RTI raises atypical measurement questions for practice, and, for this reason, it may require hybridized technical adequacy methods. Due to RTI model complexities, and the possibility of many measures and variables used over time to examine functional discrepancies in performance, decision reliability and validity questions require significant attention. Key points of analysis and recommendations for RTI technical adequacy standards are addressed, and a case study is used to illustrate technical checks. We conclude with discussion of how RTI technical adequacy may be simplified.

The Promise of Meaningful Eligibility Determination Functional Intervention-Based Multifactored Preschool Evaluation

Despite considerable controversy, standardized norm-referenced assessment procedures are widely used to determine eligibility for special education services. Under Public Law 105-17, children ages 3 through 9 may receive special services under the broad category of child with a disability—rather than traditional diagnostic labels—based on a delay in one or more developmental domains. Due to the limitations of traditional ability and developmental norm-referenced measures, the determination of developmental delays may be highly error prone and unrelated to intervention decisions. As an alternative, we describe minimal requirements for functional intervention-based assessment and suggest strategies for using these methods to analyze developmental delays and make special service eligibility decisions for preschool children (intervention-based multifactored evaluation or IBMFE). The IBMFE model provides a basis for deriving logical, natural, and meaningful discrepancies in behavior or performance through the contextual analysis of child-related, environmental, and instructional variables.

Discrepancy Score Analysis: A Computer Simulation of Classification Stability:

Although the identification of a severe discrepancy between achievement and intellectual ability has been proposed as a diagnostic indicator of learning disabilities, questions regarding the technical adequacy of severe discrepancy score classification have not been carefully examined. This study simulated the effects of measurement error, instrument selection, and the cutoff standards used to define “severe” underachievement on the stability of severe discrepancy score classification decisions. A statistical software package was used to generate aptitude and achievement data, calculate regression-prediction discrepancy scores, and classify cases of severe underachievement for a hypothetical population of 5,000 cases. The results indicated that the actuarial classification of severe underachievement was disproportionately related to chance and instrument selection. The results are generalizable to any psychometrically-based classification procedure.

Factor structure of the WISC-R for children of superior intelligence

Abstract The factor structure of the WISC-R was examined in a sample of 829 children with Full Scale IQs ≥ 120. Parallel analysis (PA) and minimum average partial (MAP) criteria were used to bracket a plausible range of solutions for the number of factors to retain. The results suggested that although there were no major factors, as many as three minor factors might be extracted. Despite some support for the replicability of a two-factor solution that generally but imprecisely conformed to the WISC-R verbal-performance dichotomy, the most parsimonious interpretation of subsequent congruence analyses favored a one-factor solution. The factor was thought to reflect “verbal” rather than “general” ability, and was consistent with the results described by Willson, Reynolds, and Gilberg (1983) for children in the standardization sample with either Verbal IQ Performance IQ or Full Scale IQ scores ≥ 120.

Ecological Foundations of Early Intervention Planned Activities and Strategic Sampling

The purpose of this article was to present strategies for establishing meaningful and coherent goals for early intervention. First, from ecological theory, the natural and empirical bases of intervention design are reviewed, and the concept of fundamental units of analysis is explicatcd. Second, the idea of planned activities, which serve as natural units for intervention decisions, is presented. Third, aspects of sampling relevant for ecological interventions are discussed. Fourth, recommendations are made for using planned activities as a basis for sampling and intervention design. Being guided by planned activities may help resolve some controversial assessment questions and lead to stronger intervention plans, due to the emphasis on fundamental natural units of analysis.

An Analysis of the Construct Validity of two Measures of Adaptive Behavior

Although the measurement of adaptive behavior has been proposed as a means of reducing diagnostic error, concerns regarding the construct validity of adaptive behavior measures have only been partly addressed. A multitrait-multimethod analysis was used to examine evidence of construct validity for the Developmental Profile and the condensed Learning Accomplishment Profile with a sample of preschool children referred for psychological evaluation. The relationship between these two measures and the Stanford-Binet was also examined. Overall, the results indicated substantial mean-score differences across instruments, a low degree of convergent and discriminant validity across adaptive behavior domains, and a high degree of within-scale intercorrelations that would disallow meaningful profile interpretation. The Developmental Profile seemed to be excessively correlated with the Binet.

Improving Instruction in Head Start Preschool Classrooms Through Feedback and Support to Teachers

This article discusses the use of data-based feedback and support to teachers in Head Start classrooms to facilitate increased use of effective instructional and managerial practices. The authors conducted a study to examine the impact of providing Head Start preschool classroom teachers with data regarding their use of established instructional and managerial practices, encouraging teacher self-selection of goals for improvement, and giving ongoing feedback and support related to goal attainment. Although the individual teacher outcomes varied, the results suggest that professional development accompanied by data-based individualized teacher feedback and support can improve teacher use of effective instructional strategies. Strengths, challenges, and implications for Head Start teachers are discussed.

The Devereux Adolescent Behavior Rating Scale: A tentative model of second-order factor structure across independent clinical samples

The second-order factor structure of the Devereux Adolescent Behavior Rating Scale was examined in (a) a sample of 254 adolescents who were receiving residential treatment for severe emotional/behavior disorders and (b) an independent sample of 404 adolescents hospitalized for substance abuse. A plausible range of factors was estimated for each group through parallel and average partial analyses and suggested wither a two- or three-factor solution. Subsequent congruence analyses provided tentative support for a three-factor model: (a) undercontrolled/disruptive behaviors; (b) withdrawn/psychotic behaviors; and (c) needs approval/dependent behaviors. The third factor was poorly defined in the residential treatment sample, but extraction of the third factor increased the across-group replicability of the first two factors.