Spyros Konstantopoulos

Fixed Effects and Variance Components Estimation in Three-Level Meta-Analysis

Meta-analytic methods have been widely applied to education, medicine, and the social sciences. Much of meta-analytic data are hierarchically structured because effect size estimates are nested within studies, and in turn, studies can be nested within level-3 units such as laboratories or investigators, and so forth. Thus, multilevel models are a natural framework for analyzing meta-analytic data. This paper discusses the application of a Fisher scoring method in two-level and three-level meta-analysis that takes into account random variation at the second and third levels. The usefulness of the model is demonstrated using data that provide information about school calendar types. sas proc mixed and hlm can be used to compute the estimates of fixed effects and variance components. Copyright

Comparing the Impact of Online and Face-to-Face Professional Development in the Context of Curriculum Implementation

This study employed a randomized experiment to examine differences in teacher and student learning from professional development (PD) in two modalities: online and face-to-face. The study explores whether there are differences in teacher knowledge and beliefs, teacher classroom practice, and student learning outcomes related to PD modality. Comparison of classroom practice and student learning outcomes, normally difficult to establish in PD research, is facilitated by the use of a common set of curriculum materials as the content for PD and subsequent teaching. Findings indicate that teachers and students exhibited significant gains in both conditions, and that there was no significant difference between conditions. We discuss implications for the delivery of teacher professional learning.

What Are the Long-Term Effects of Small Classes on the Achievement Gap? Evidence from the Lasting Benefits Study.

The findings on the social distribution of the immediate and lasting benefits of small classes have been mixed. We used data from Project STAR and the Lasting Benefits Study to examine the long‐term effects of small classes on the achievement gap in mathematics, reading, and science scores (Stanford Achievement Test). The results consistently indicated that all types of students benefit more in later grades from being in small classes in early grades. These positive effects are significant through grade 8. Longer periods in small classes produced higher increases in achievement in later grades for all types of students. For certain grades, in reading and science, low achievers seem to benefit more from being in small classes for longer periods. It appears that the lasting benefits of the cumulative effects of small classes may reduce the achievement gap in reading and science in some of the later grades.

The Persistence of Teacher Effects in Elementary Grades

Results from experimental and nonexperimental studies have shown that teachers differ in their effectiveness. In addition, evidence from nonexperimental studies has indicated that teacher effects last for 3 years in elementary grades. This study uses data from Project STAR and its follow-up study, the Lasting Benefits Study, to examine whether teacher effects from kindergarten to fifth grade can simultaneously affect sixth grade achievement. Teacher effects are defined as teacher-specific residuals adjusted for student background and treatment effects. Findings indicate that the teacher effects persist through sixth grade in mathematics, reading, and science. The findings also suggest that teacher effects are important and that their cumulative effects on student achievement are considerable.

Do the Disadvantaged Benefit More from Small Classes? Evidence from the Tennessee Class Size Experiment

The effects of class size on academic achievement have been studied for decades. Although recent research from randomized experiments points to positive effects of small classes, the evidence about the social distribution of effects is less clear. Some scholars have contended that the effects of small classes are larger for minorities and the disadvantaged. These claims have led to policy decisions to implement small classes to reduce inequality in educational outcomes. This article investigates possible differential effects of small classes on achievement using data from Project STAR, a four-year, large-scale randomized experiment on the effects of class size. The small class effects on both reading and mathematics achievement are somewhat larger for minorities and low socioeconomic status (SES) students. However, the differential effects for minority students (interactions) are statistically significant only for reading achievement in one of the models examined, but not others. None of the differential effects of small classes for low SES students are statistically significant. Thus, while there are unambiguous positive effects of small classes on both reading and mathematics achievement, there is no evidence of differential effects for low SES students and only weak evidence of differential effects for minority students in reading achievement.

Separate but correlated: The latent structure of space and mathematics across development.

The relations among various spatial and mathematics skills were assessed in a cross-sectional study of 854 children from kindergarten, third, and sixth grades (i.e., 5 to 13 years of age). Children completed a battery of spatial mathematics tests and their scores were submitted to exploratory factor analyses both within and across domains. In the within domain analyses, all of the measures formed single factors at each age, suggesting consistent, unitary structures across this age range. Yet, as in previous work, the 2 domains were highly correlated, both in terms of overall composite score and pairwise comparisons of individual tasks. When both spatial and mathematics scores were submitted to the same factor analysis, the 2 domain specific factors again emerged, but there also were significant cross-domain factor loadings that varied with age. Multivariate regressions replicated the factor analysis and further revealed that mental rotation was the best predictor of mathematical performance in kindergarten, and visual-spatial working memory was the best predictor of mathematical performance in sixth grade. The mathematical tasks that predicted the most variance in spatial skill were place value (K, 3rd, 6th), word problems (3rd, 6th), calculation (K), fraction concepts (3rd), and algebra (6th). Thus, although spatial skill and mathematics each have strong internal structures, they also share significant overlap, and have particularly strong cross-domain relations for certain tasks. (PsycINFO Database Record

Teacher Effects on Minority and Disadvantaged Students’ Grade 4 Achievement

ABSTRACT The authors examined the differential effects of teachers on female, minority, and low-socioeconomic status (SES) students’ achievement in Grade 4. They used data from a randomized experiment (Project STAR) and its follow-up study (LBS). Student outcomes included Grade 4 SAT scores in mathematics, reading, and science and student demographics included gender, race, and SES. The authors used multilevel models to determine how teacher effectiveness interacted with student gender, race, and SES. We also explored whether teacher effects were more pronounced in schools with high proportions of minority or female students. Results indicated that all students benefited from having effective teachers. The differential teacher effects on female, minority, and low-SES students’ achievement, however, were insignificant. There is some evidence in mathematics that teacher effects are more pronounced in high-minority schools. Finally, teacher effects seem to be consistent within and between schools.

Is the Persistence of Teacher Effects in Early Grades Larger for Lower-Performing Students?

We examined the persistence of teacher effects from grade to grade on lower-performing students using data from Project STAR. Teacher effects were computed as residual classroom achievement within schools. Teacher effects in one grade predicted achievement in following grades using quantile regression. Results consistently indicated that all students benefited similarly from teachers, and differential teacher effects were not evident. Overall, lower-performing students benefit as much as other students from teachers except in fourth grade, where lower-performing students benefit more. Having effective teachers in successive grades seems beneficial to lower-performing students in mathematics and reading. However, having low-effective teachers in successive grades is detrimental to all students especially in mathematics.

The Impact of Covariates on Statistical Power in Cluster Randomized Designs: Which Level Matters More?

Field experiments with nested structures are becoming increasingly common, especially designs that assign randomly entire clusters such as schools to a treatment and a control group. In such large-scale cluster randomized studies the challenge is to obtain sufficient power of the test of the treatment effect. The objective is to maximize power without adding many clusters that make the study much more expensive. In this article I discuss how power estimates of tests of treatment effects in balanced cluster randomized designs are affected by covariates at different levels. I use third-grade data from Project STAR, a field experiment about class size, to demonstrate how covariates that explain a considerable proportion of variance in outcomes increase power significantly. When lower level covariates are group-mean centered and clustering effects are larger, top-level covariates increase power more than lower level covariates. In contrast, when clustering effects are smaller and lower level covariates are grand-mean centered or uncentered, lower level covariates increase power more than top-level covariates.

Optimal sampling of units in three-level cluster randomized designs: An ancova framework

Field experiments with nested structures assign entire groups such as schools to treatment and control conditions. Key aspects of such cluster randomized experiments include knowledge of the intraclass correlation structure and the sample sizes necessary to achieve adequate power to detect the treatment effect. The units at each level of the hierarchy have a cost associated with them, however, and thus, researchers need to take budget and costs into account when designing their studies. This article uses analysis of covariance and provides methods for computing power within an optimal design framework that incorporates costs of units at different levels and covariate effects for three-level cluster randomized balanced designs. The optimal sample sizes are a function of the variances at each level and the cost of each unit. Overall, the results suggest that when units at higher levels become more expensive, the researcher should sample units at lower levels. The covariates affect the sampling of units and the power estimates. Fewer units need to be sampled at levels where covariates explain considerable proportions of the variance.