Deborah K. Reed

Retell as an Indicator of Reading Comprehension

The purpose of this narrative synthesis is to determine the reliability and validity of retell protocols for assessing reading comprehension of students in grades K–12. Fifty-four studies were systematically coded for data related to the administration protocol, scoring procedures, and technical adequacy of the retell component. Retell was moderately correlated with standardized measures of reading comprehension and, with older students, had a lower correlation with decoding and fluency. Literal information was retold more frequently than inferential, and students with learning disabilities or reading difficulties needed more supports to demonstrate adequate recall. Great variability was shown in the prompting procedures, but scoring methods were more consistent across studies. The influences of genre, background knowledge, and organizational features were often specific to particular content, texts, or students. Overall, retell has not yet demonstrated adequacy as a progress monitoring instrument.

A Synthesis of Peer-Mediated Academic Interventions for Secondary Struggling Learners

A synthesis of the extant research on peer-mediated reading and math interventions for students in regular or alternative education settings with academic difficulties and disabilities in Grades 6 to 12 (ages 11–18) is presented. Interventions conducted between 2001 and 2012 targeting reading and math were included if they measured effects on at least one academic outcome measure. A total of 13 intervention studies were synthesized in which 10 studies employed an experimental or quasi-experimental design and three studies used a single-case design. Findings from the 13 studies revealed mostly moderate to high effects favoring peer mediation, particularly when implementing a peer-mediated feedback component. In addition, findings suggest such interventions have social validity among adolescents and teachers. More rigorous research on secondary peer-mediated math interventions, peer-mediated interventions in alternative settings, and effective ways to pair dyads to incorporate a structured feedback component is warranted. Implications for peer-mediated instruction for academically struggling adolescents are discussed.

A Synthesis of Professional Development on the Implementation of Literacy Strategies for Middle School Content Area Teachers.

Abstract This paper synthesized studies of professional development for middle school content area teachers and the teachers’ subsequent implementation of literacy strategies. Four studies were identified as having a majority of participants teaching English/reading, mathematics, science, and social studies in grades 6 through 8. Articles meeting the criteria included two qualitative studies of the impact of professional development on implementation of literacy strategies, one ethnographic study of the characteristics of content area teachers with strong implementation of literacy strategies, and one quasiexperimental study of the impact of professional development on student reading performance. Findings indicate that ongoing schoolwide initiatives that are responsive to teachers’ perceived needs hold promise for increasing literacy instruction across the curriculum and improving some student reading skills.

An Examination of Assessment Fidelity in the Administration and Interpretation of Reading Tests

Researchers have expressed concern about implementation fidelity in intervention research but have not extended that concern to assessment fidelity, or the extent to which pre-/posttests are administered and interpreted as intended. When studying reading interventions, data gathering heavily influences the identification of students, the curricular components delivered, and the interpretation of outcomes. However, information on assessment fidelity is rarely reported. This study examined the fidelity with which individuals paid to be testers for research purposes were directly observed administering and interpreting reading assessments for middle school students. Of 589 testing packets, 45 (8% of the total) had to be removed from the data set for significant abnormalities and another 484 (91% of the remaining packets) had correctable errors only found in double scoring. Results indicate reading assessments require extensive training, highly structured protocols, and ongoing calibration to produce reliable and valid results useful in applied research.

A Synthesis of Research on Informational Text Reading Interventions for Elementary Students With Learning Disabilities

This research synthesis was conducted to understand the effectiveness of interventions designed to improve learning from informational text for students with learning disabilities in elementary school (K–5). The authors identified 18 studies through a comprehensive search. The interventions were evaluated to determine treatment effects and to understand implementation and methodological variables that influenced outcomes. Moderate to large effect sizes on researcher-developed measures for cognitive strategy interventions were reported. Interventions that utilized graphic organizers as study guides to support social studies learning were also associated with improved outcomes. The findings are considered within the context of limited implementation of standardized measures. The authors extend findings from previous research by reporting a paucity of interventions to enhance higher-level cognitive and comprehension skills. The majority of reviewed studies targeted fact acquisition and main idea identification, and overall encouraging findings were noted for these skills. Implications for future research are discussed.

“Our Teachers … Don’t Give Us No Help, No Nothin’”: Juvenile Offenders’ Perceptions of Academic Support

This qualitative study investigated the educational experiences that shaped juvenile offenders’ perceptions of the academic support they received in juvenile justice (JJ) settings and their public schools. We conducted interviews with students and JJ school personnel, two focus groups, classroom observations, and student surveys. Juvenile offenders characterized their teachers as supportive or unsupportive based on whether they made proactive efforts to help students learn. Disconnection with school occurred when students perceived teachers did not care and when there were difficulties in earning or transferring credits. Findings suggest that juvenile offenders demonstrate academic resiliency and a continued desire to pursue education.

The Ecological and Population Validity of Reading Interventions for Adolescents: Can Effectiveness Be Generalized?.

This article examined the ecological and population validity of research on reading interventions for adolescents in Grades 6 through 12. The 26 studies meeting selection criteria were analyzed to determine the characteristics of the students, interventionists, classroom structures, and school environments used, as well as whether there were differential effects of treatments across those characteristics. In the 20+ years since the calls by the Council for Learning Disabilities and National Joint Committee on Learning Disabilities for greater specificity in descriptions of study participants and contexts, the findings of this study suggest that researchers have provided greater detail on participants, but many questions remain about the extent to which findings can be generalized. Specifically, gaps in the research exist with regard to African American and Native American students; English language learners; students in suburban, rural, and adjudicated schools; students in high school; interventions delivered by regular classroom teachers; interventions focused on vocabulary; and interventions in large groups and general education classrooms.

The Influence of Testing Prompt and Condition on Middle School Students’ Retell Performance

This study examined whether the type of prompt or the method of passage reading had an effect on the retell performance of 6th–8th graders randomly assigned to one of four retell testing conditions. Both the type of prompt and the use of follow-up prompting were significantly related to the percentage of predetermined idea units retold. Effect sizes were approximately moderate (d = .44–.62) when one change was made to the prompt but were strong (d = .96–1.05) with a combination of changes. The addition of silent reading did not significantly improve performance.

Assessment Fidelity in Reading Intervention Research: A Synthesis of the Literature

Recent studies indicate that examiners make a number of intentional and unintentional errors when administering reading assessments to students. Because these errors introduce construct-irrelevant variance in scores, the fidelity of test administrations could influence the results of evaluation studies. To determine how assessment fidelity is being addressed in reading intervention research, we systematically reviewed 46 studies conducted with students in Grades K–8 identified as having a reading disability or at-risk for reading failure. Articles were coded for features such as the number and type of tests administered, experience and role of examiners, tester to student ratio, initial and follow-up training provided, monitoring procedures, testing environment, and scoring procedures. Findings suggest assessment integrity data are rarely reported. We discuss the results in a framework of potential threats to assessment fidelity and the implications of these threats for interpreting intervention study results.

The Validity of a Holistically Scored Retell Protocol for Determining the Reading Comprehension of Middle School Students

In this study, the authors examined the validity of a holistically scored retell within a confirmatory factor analysis framework by comparing the fit of a three-factor model of reading with the data from a diverse sample of seventh and eighth graders. The final model demonstrated adequate fit, χ2(32) = 97.316; comparative fit index = .96; Tucker–Lewis index = .94; and root mean square error of approximation = .08. Retell’s chi-square difference, Δχ2(1) = 16.652, p < .001, and factor loading (.250, p < .001) were higher for the comprehension construct. Similarly, retell’s correlation to comprehension measures (r = .155–.257, p < .01) was stronger than its relationship to measures of fluency (r = .158–.183, p < .01) or word identification (r = .132, p < .05). However, retell had a large residual variance (.938) and low interrater reliability (κ = .37), suggesting that improvements to the instrument are needed. Despite overall latent differences, retell did not demonstrate differential item functioning.