James J. Appleton

Meta-Analytic Review of Responsiveness-to-Intervention Research: Examining Field-Based and Research-Implemented Models.

A responsiveness-to-intervention (RTI) approach to diagnosing LD is a leading alter native to current practice. This study con ducted a meta-analytic review of research on four existing large-scale RTI models and other models implemented for research. Twenty-four effect sizes and unbiased esti mates of effect (UEE) were computed. Results found a larger UEE for studies of existing RTI models than those implemented by university faculty for research, but both were strong. The UEE for student achievement and sys temic outcomes both exceeded 1.0, but the UEE for systemic outcomes among field-based RTI models was nearly twice as large as for stu dent outcomes. Further, RTI models imple mented for research led to an UEE of 1.14 for student outcomes and 0.47 for systemic out comes. The mean percentage of nonrespon ders in the studies was 19.8% (SD= 12.5), and an average of 1.68% (SD = 1.45) of the stu dent population was placed into special edu cation. Implications for practice and future research are included.

Measuring Student Engagement among Elementary Students: Pilot of the Student Engagement Instrument--Elementary Version.

Early school withdrawal, commonly referred to as dropout, is associated with a plethora of negative outcomes for students, schools, and society. Student engagement, however, presents as a promising theoretical model and cornerstone of school completion interventions. The purpose of the present study was to validate the Student Engagement Instrument-Elementary Version (SEI-E). The psychometric properties of this measure were assessed based on the responses of an ethnically diverse sample of 1,943 students from an urban locale. Exploratory and confirmatory factor analyses indicated that the 4-factor model of student engagement provided the best fit for the current data, which is divergent from previous SEI studies suggesting 5- and 6-factor models. Discussion and implications of these findings are presented in the context of student engagement and dropout prevention.

Technical Education Curriculum Assessment.

The purpose of this paper is to describe and determine the efficacy of a Technical Education Curriculum Assessment (TECA). The TECA was designed to guide the judgment of the quality of technical education curricular materials. Three research strands were combined into a theoretical framework which underlies the education of effective technicians. The TECA consists of sets of rubrics which focus on workplace competencies, technical accuracy, and pedagogical soundness. The rubrics were constructed using a deductive-inductive approach. This was an iterative process that ensured validity by moving back and forth from the theoretical framework uncovered in the literature review (deductive) to the application of the rubrics to actual curricular materials (inductive). We describe the process of rubrics development and provide data which support their validity and reliability. This psychometrically sound instrument should assist industry and education professionals to make more informed decisions when designing, implementing, and evaluating technical education curriculum.

An Examination of the Validity of the Behavioral and Emotional Screening System in a Rural Elementary School Validity of the BESS

The purpose of the current study was to evaluate a screening instrument. The sample contained 496 elementary children from the rural southeast. Properties of the Teacher, Parent, and Student Forms of the Behavioral and Emotional Screening System were examined. Results indicated that all forms had high levels of internal consistency. There were low to moderate correlations between the Teacher Form and reading, behavior, and attendance variables. The Parent Form was significantly correlated with attendance and reading. The Student Forms were significantly correlated with office discipline referrals, attendance, and reading. The Teacher Form identified 28% of students as being at-risk (elevated or extremely elevated), followed by 20% and 32% on the Parent and Student Forms, respectively. The risk groups were significantly different in terms of attendance, behavior, and achievement. However, interrater agreement regarding risk status across different raters was low. All forms demonstrated low positive predictive power and low to moderate sensitivity relative to an outcome of office discipline referrals but did show reasonably strong negative predictive power and specificity. Future directions for research are presented.

Concurrent and Predictive Validity of the Student Engagement Instrument.

The Student Engagement Instrument (SEI) is a self-report measure of cognitive and affective engagement with school. Prior SEI validation studies have focused primarily on construct validity through analyses of internal consistency, factor analysis, and measurement invariance. Results are presented here from a two-pronged study of the criterion validity of SEI scores. Using a middle school sample (N = 35,900), concurrent validity was assessed through analyses of group differences in SEI scores across student subgroups expected to differ in cognitive and affective engagement levels: behaviorally disengaged versus non-disengaged, high-risk versus low-risk disability status, and high versus low academic achievement. Next, through multiple logistic regression analyses, the 4-year predictive validity of SEI scores for on-time graduation and dropout was assessed in a cohort of first-time ninth graders (N = 11,588). Nearly all SEI factors demonstrated directionally consistent associations with each criterion, including considerable long-term predictive associations with both dropout and on-time graduation.

An Examination of the Validity of Two Measures of Student Engagement.

This study evaluated the psychometric properties of two measures of student engagement, the Student Engagement Instrument (SEI) and the Motivation-Engagement Scale (MES), with adolescents in the southeastern United States. Confirmatory factor analyses revealed an acceptable fit of the SEI and a relatively poor fit of the MES in this sample. Correlational analyses provided evidence of convergent and divergent validity of SEI and MES factors. In addition, SEI and MES factors were correlated as expected with external measures of academic functioning and school behavior.

Curriculum-Based Measurement as a Predictor of Performance on a State Assessment: Diagnostic Efficiency of Local Norms

With the concurrent emphasis on accountability, prevention, and early intervention, curriculum-based measurement of reading (R-CBM) is playing an increasingly important role in the educational process. This study investigated the differences in diagnostic accuracy and utility between commercial norms and local norms when making high-stakes, local decisions. Scores on Dynamic Indicators of Early Literacy Skills Oral Reading Fluency for 1,374 students in Grades 2 to 5 were used to predict outcomes the Georgia reading achievement test, the Criterion Referenced Competency Tests. Local norms were generated using logistic regression and receiver operator characteristic curve analysis. The generated cut scores were compared to the commercial norms for differences in diagnostic efficiency. The generated cut scores were lower than the commercial norms and had improved diagnostic efficiency. Implications related to educational policy and the use of R-CBM are discussed.

Beyond School Records: The Value of Cognitive and Affective Engagement in Predicting Dropout and On-Time Graduation

Early warning systems use school record data— such as attendance rate, behavior records, and course performance—to identify students at risk of dropping out. These are useful predictors of graduation-related outcomes, in large part because they indicate a students level of engagement with school. However, these data do not indicate how invested students are in education—information that could help school counselors and other staff understand and intervene when students are falling off the path to graduation. To examine whether student engagement surveys have additional predictive value beyond data readily available in school databases, we followed a cohort of students, who completed a survey of cognitive/affective engagement as ninth graders, to one year beyond their expected high school graduation. Some engagement factors measured by the survey met rigorous tests of predictive value in terms of identifying which students were falling off the graduation path, even when controlling for other powerful predictors of the outcome.