Larry V. Hedges

Distribution Theory for Glass's Estimator of Effect Size and Related Estimators.

Glasss estimator of effect size, the sample mean difference divided by the sample standard deviation, is studied in the context of an explicit statistical model. The exact distribution of Glasss estimator is obtained and the estimator is shown to have a small sample bias. The minimum variance unbiased estimator is obtained and shown to have uniformly smaller variance than Glasss (biased) estimator. Measurement error is shown to attenuate estimates of effect size and a correction is given. The effects of measurement invalidity are discussed. Expressions for weights that yield the most precise weighted estimate of effect size are also derived.

Fixed- and Random-Effects Models in Meta-Analysis

There are 2 families of statistical procedures in meta-analysis: fixed- and randomeffects procedures. They were developed for somewhat different inference goals: making inferences about the effect parameters in the studies that have been observed versus making inferences about the distribution of effect parameters in a population of studies from a random sample of studies. The authors evaluate the performance of confidence intervals and hypothesis tests when each type of statistical procedure is used for each type of inference and confirm that each procedure is best for making the kind of inference for which it was designed. Conditionally random-effects procedures (a hybrid type) are shown to have properties in between those of fixed- and random-effects procedures. The use of quantitative methods to summarize the results of several empirical research studies, or metaanalysis, is now widely used in psychology, medicine, and the social sciences. Meta-analysis usually involves describing the results of each study by means of a numerical index (an estimate of effect size, such as a correlation coefficient, a standardized mean difference, or an odds ratio) and then combining these estimates across studies to obtain a summary. Two somewhat different statistical models have been developed for inference about average effect size from a collection of studies, called the fixed-effects and random-effects models. (A third alternative, the mixedeffects model, arises in conjunction with analyses involving study-level covariates or moderator variables, which we do not consider in this article; see Hedges, 1992.) Fixed-effects models treat the effect-size parameters as fixed but unknown constants to be estimated and usually (but not necessarily) are used in conjunction with assumptions about the homogeneity of effect parameters (see, e.g., Hedges, 1982; Rosenthal & Rubin, 1982). Random-effects models treat the effectsize parameters as if they were a random sample from

THE META‐ANALYSIS OF RESPONSE RATIOS IN EXPERIMENTAL ECOLOGY

Meta-analysis provides formal statistical techniques for summarizing the results of independent experiments and is increasingly being used in ecology. The response ratio (the ratio of mean outcome in the experimental group to that in the control group) and closely related measures of proportionate change are often used as measures of effect magnitude in ecology. Using these metrics for meta-analysis requires knowledge of their statistical properties, but these have not been previously derived. We give the approximate sampling distribution of the log response ratio, discuss why it is a particularly useful metric for many applications in ecology, and demonstrate how to use it in meta-analysis. The meta-analysis of response-ratio data is illustrated using experimental data on the effects of increased atmospheric CO2 on plant biomass responses.

The Effect of School Resources on Student Achievement

A universe of education production function studies was assembled in order to utilize meta-analytic methods to assess the direction and magnitude of the relations between a variety of school inputs and student achievement. The 60 primary research studies aggregated data at the level of school districts or smaller units and either controlled for socioeconomic characteristics or were longitudinal in design. The analysis found that a broad range of resources were positively related to student outcomes, with effect sizes large enough to suggest that moderate increases in spending may be associated with significant increases in achievement. The discussion relates the findings of this study with trends in student achievement from the National Assessment of Educational Progress and changes in social capital over the last two decades.

How Large Are Teacher Effects

It is widely accepted that teachers differ in their effectiveness, yet the empirical evidence regarding teacher effectiveness is weak. The existing evidence is mainly drawn from econometric studies that use covariates to attempt to control for selection effects that might bias results. We use data from a four-year experiment in which teachers and students were randomly assigned to classes to estimate teacher effects on student achievement. Teacher effects are estimated as between-teacher (but within-school) variance components of achievement status and residualized achievement gains. Our estimates of teacher effects on achievement gains are similar in magnitude to those of previous econometric studies, but we find larger effects on mathematics achievement than on reading achievement. The estimated relation of teacher experience with student achievement gains is substantial, but is statistically significant only for 2nd-grade reading and 3rd-grade mathematics achievement. We also find much larger teacher effect variance in low socioeconomic status (SES) schools than in high SES schools.

A basic introduction to fixed‐effect and random‐effects models for meta‐analysis

There are two popular statistical models for meta-analysis, the fixed-effect model and the random-effects model. The fact that these two models employ similar sets of formulas to compute statistics, and sometimes yield similar estimates for the various parameters, may lead people to believe that the models are interchangeable. In fact, though, the models represent fundamentally different assumptions about the data. The selection of the appropriate model is important to ensure that the various statistics are estimated correctly. Additionally, and more fundamentally, the model serves to place the analysis in context. It provides a framework for the goals of the analysis as well as for the interpretation of the statistics. In this paper we explain the key assumptions of each model, and then outline the differences between the models. We conclude with a discussion of factors to consider when choosing between the two models. Copyright

An Exchange: Part I*: Does Money Matter? A Meta-Analysis of Studies of the Effects of Differential School Inputs on Student Outcomes

Research on educational production functions attempts to model the relation between resource inputs and school outcomes such as educational achievement. Over the last decade a series of influential reviews of this literature have suggested that there is no systematic relation between resource inputs and school outcomes when controlling for student characteristics such as socioeconomic status. The inference procedure used in these reviews, vote counting, is known to be problematic. This study is a reanalysis of data from these earlier reviews, using more sophisticated synthesis methods. It shows systematic positive relations between resource inputs and school outcomes. Moreover, analyses of the magnitude of these relations suggest that the median relation (regression coefficient) is large enough to be of practical importance.While this reanalysis suggests that previous data do not support the conclusions that Hanushek and others derived from it, limitations of their data set warrant caution in using it for policy formation.

STATISTICAL ISSUES IN ECOLOGICAL META‐ANALYSES

Meta-analysis is the use of statistical methods to summarize research findings across studies. Special statistical methods are usually needed for meta-analysis, both because effect-size indexes are typically highly heteroscedastic and because it is desirable to be able to distinguish between-study variance from within-study sampling-error variance. We outline a number of considerations related to choosing methods for the meta-analysis of ecological data, including the choice of parametric vs. resampling methods, reasons for conducting weighted analyses where possible, and comparisons fixed vs. mixed models in categorical and regression-type analyses.

Categories and Particulars: Prototype Effects in Estimating Spatial Location.

A model of category effects on reports from memory is presented. The model holds that stimuli are represented at 2 levels of detail: a fine-grain value and a category. When memory is inexact but people must report an exact value, they use estimation processes that combine the remembered stimulus value with category information. The proposed estimation processes include truncation at category boundaries and weighting with a central (prototypic) category value. These processes introduce bias in reporting even when memory is unbiased, but nevertheless may improve overall accuracy (by decreasing the variability of reports). Four experiments are presented in which people report the location of a dot in a circle. Subjects spontaneously impose horizontal and vertical boundaries that divide the circle into quadrants. They misplace dots toward a central (prototypic) location in each quadrant, as predicted by the model. The proposed model has broad implications; notably, it has the potential to explain biases of the sort described in psychophysics (contraction bias and the bias captured by Webers law) as well as symmetries in similarity judgments, without positing distorted representations of physical scales.

Estimation of a Single Effect Size: Parametric and Nonparametric Methods

This chapter focuses on the study of parametric and nonparametric methods for estimating the effect size (standardized mean difference) from a single experiment. It is important to recognize that estimating and interpreting a common effect size is based on the belief that the population effect size is actually the same across studies. Otherwise, estimating a mean effect may obscure important differences between the studies. The chapter discusses several alternative point estimators of the effect size δ from a single two-group experiment. These estimators are based on the sample standardized mean difference but differ by multiplicative constants that depend on the sample sizes involved. Although the estimates have identical large sample properties, they generally differ in terms of small sample properties. The statistical properties of estimators of effect size depend on the model for the observations in the experiment. A convenient and often realistic model is to assume that the observations are independently normally distributed within groups of the experiment.