Howard Wainer

Estimating Coefficients in Linear Models: It Don't Make No Nevermind

It is proved that under very general circumstances coefficients in multiple regression models can be replaced with equal weights with almost no loss in accuracy on the original data sample. It is then shown that these equal weights will have greater robustness than least squares regression coefficients. The implications for problems of prediction are discussed. In the two decades since Meehls (1954) book on the respective accuracy of clinical versus clerical prediction, little practical consequence has been observed. Diagnoses are still made by clinicians, not by clerks; college admissions are still done by committee, not by computer. This is true despite the considerable strength of Meehls argument that humans are very poor at combining information optimally and that regression models evidently combine information rather well. These points were underlined in some recent work by Dawes and Corrigan (1974), in which they found again that human predictors do poorly when compared with regression models. Strikingly, they found that for some reason, linear models with random regression weights also do better than do humans. Even more striking, when all regression weights were set equal to one another they found still higher correlation with criterion on a validating sample. The obvious question here is Why? Is it because humans are so terrible at combining information that almost any rule works better, or is it some artifact of linear regression?

The Rasch model as additive conjoint measurement

The object of this paper is to present Raschs psychometric model as a special case of additive conjoint measurement. The connection between these two areas has been discussed before, but largely ignored. Because the theory of conjoint measurement has been formulated determinis tically, there have been some difficulties in its application. It is pointed out in this paper that the Rasch model, which is a stochastic model, does not suffer from this fault. The exposition centers on the analyses of two data sets, each of which was ana lyzed using Rasch scaling methods as well as some of the methods of conjoint measurement. The results, using the different procedures, are com pared.

Understanding Graphs and Tables

Quantitative phenomena can be displayed effectively in a variety of ways, but to do so requires an understanding of both the structure of the phenomena and the limitations of candidate display formats. This article (a) re-counts three historic instances of the vital role data displays played in important discoveries, (b) provides three levels of information that form the basis of a theory of display to help us better measure both display quality and human graphicacy, and (c) describes three steps to improve the quality of tabular presentation.

How to Display Data Badly

Abstract Methods for displaying data badly have been developing for many years, and a wide variety of interesting and inventive schemes have emerged. Presented here is a synthesis yielding the 12 most powerful techniques that seem to underlie many of the realizations found in practice. These 12 (the dirty dozen) are identified and illustrated.

Drawing inferences from self-selected samples

As known, adventure and experience about lesson, entertainment, and knowledge can be gained by only reading a book. Even it is not directly done, you can know more about this life, about the world. We offer you this proper and easy way to gain those all. We offer many book collections from fictions to science at all. One of them is this drawing inferences from self selected samples that can be your partner.

A General Bayesian Model for Testlets: Theory and Applications

The need for more realistic and richer forms of assessment in educational tests has led to the inclusion (in many tests) of polytomously scored items, multiple items based on a single stimulus (a “testlet”), and the increased use of a generalized mixture of binary and polytomous item formats. In this paper, the authors extend earlier work on the modeling of testlet-based response data to include the situation in which a test is composed, partially or completely, of polytomously scored items and/or testlets. The model they propose, a modified version of commonly employed item response models, is embedded within a fully Bayesian framework, and inferences under the model are obtained using Markov chain Monte Carlo techniques. The authors demonstrate its use within a designed series of simulations and by analyzing operational data from the North Carolina Test of Computer Skills and the Educational Testing Service’s Test of Spoken English. Their empirical findings suggest that the North Carolina Test of Computer Skills exhibits significant testlet effects, indicating significant dependence of item scores obtained from common stimuli, whereas the Test of Spoken English does not.

Some standard errors in item response theory

The mathematics required to calculate the asymptotic standard errors of the parameters of three commonly used logistic item response models is described and used to generate values for some common situations. It is shown that the maximum likelihood estimation of a lower asymptote can wreak havoc with the accuracy of estimation of a location parameter, indicating that if one needs to have accurate estimates of location parameters (say for purposes of test linking/equating or computerized adaptive testing) the sample sizes required for acceptable accuracy may be unattainable in most applications. It is suggested that other estimation methods be used if the three parameter model is applied in these situations.

Adjusting for Differential Base Rates: Lord's Paradox Again

When the responses of 2 or more groups to the relative effects of some stimulus are compared, it is often important to adjust statistically the estimates of those effects for baseline differences among those groups. This is often the case in experiments on heart rate for animals of different ages. How should such adjustment be done? Among the competing methodologies are (a) subtract the base rate, (b) divide by the base rate, and (c) covary out the base rate. Because each can give a different answer, the choice is crucial. This article shows that this is an example of Lords Paradox and that Rubins Model for the measurement of causal effects allows researchers to understand what the assumptions are underlying the validity of each adjustment strategy. The answer for heart rate data is almost surely Methodology (a).

Testlet Response Theory: An Analog for the 3PL Model Useful in Testlet-Based Adaptive Testing

The invention of short multiple choice test items provided an enormous technical and practical advantage for test developers; certainly the items could be scored easily, but that was just one of the reasons for their popular adoption in the early part of the 20th century. A more important reason was the increase in validity offered because of the speed with which such items could be answered. This meant that a broad range of content specifications could be addressed, and hence an examinee need no longer be penalized because of an unfortunate choice of constructed response (e.g., essay) question. These advantages, as well as many others (see Anastasi, 1976, 415-417) led the multiple choice format to become, by far, the dominant form used in large-scale standardized mental testing throughout this century. Nevertheless, this breakthrough in test construction, dominant at least since the days of Army is currently being reconsidered. Critics of tests that are made up of large numbers of short questions suggest that decontextualized items yield a task that is abstracted too far from the domain of inference for many potential uses. For several reasons, only one of them as a response to this criticism, variations in test theory were considered that would allow the retention of the shortanswer format while at the same time eliminating the shortcomings expressed by those critics. One of these variations was the development of item response theory (IRT), an analytic breakthrough in test scoring. A key feature of IRT is that examinee responses are conceived of as reflecting evidence of a particular location on a single underlying latent

Robust Statistics: A Survey and Some Prescriptions

It is noted that the usual estimators that are optimal under a Gaussian assumption are very vulnerable to the effects of outliers. A survey of robust alternatives to the mean, standard deviation, product moment correlation, t-test, and analysis of variance is offered. Robust methods of factor analysis, principal components analysis and multivariate analysis of variance are also surveyed, as are schemes for outlier detection.