Bruce Williams

Fitting polytomous item response theory models to multiple-choice tests

This study examined how well current software implementations of four polytomous item response theory models fit several multiple-choice tests. The models were Bocks (1972) nominal model, Samejimas (1979) multiple-choice Model C, Thissen & Steinbergs (1984) multiple-choice model, and Levines (1993) maximum-likelihood formula scoring model. The parameters of the first three of these models were estimated with Thissens (1986) MULTILOG computer program; Williams & Levines (1993) FORSCORE program was used for Levines model. Tests from the Armed Services Vocational Aptitude Battery,the Scholastic Aptitude Test, and the American College Test Assessment were analyzed. The models were fit in estimation samples of approximately 3,000; cross-validation samples of approximately 3,000 were used to evaluate goodness of fit. Both fit plots and X2 statistics were used to determine the adequacy of fit. Bocks model provided surprisingly good fit; adding parameters to the nominal model did not yield improvements in fit. FORSCORE provided generally good fit for Levines nonparametric model across all tests. Index terms: Bocks nominal model, FORSCORE, maximum likelihood formula scoring, MULTILOG, polytomous IRT.

Fitting Item Response Theory Models to Two Personality Inventories: Issues and Insights.

The present study compared the fit of several IRT models to two personality assessment instruments. Data from 13,059 individuals responding to the US-English version of the Fifth Edition of the Sixteen Personality Factor Questionnaire (16PF) and 1,770 individuals responding to Goldbergs 50 item Big Five Personality measure were analyzed. Various issues pertaining to the fit of the IRT models to personality data were considered. We examined two of the most popular parametric models designed for dichotomously scored items (i.e., the two- and three-parameter logistic models) and a parametric model for polytomous items (Samejimas graded response model). Also examined were Levines nonparametric maximum likelihood formula scoring models for dichotomous and polytomous data, which were previously found to provide good fits to several cognitive ability tests (Drasgow, Levine, Tsien, Williams, & Mead, 1995). The two- and three-parameter logistic models fit some scales reasonably well but not others; the graded response model generally did not fit well. The nonparametric formula scoring models provided the best fit of the models considered. Several implications of these findings for personality measurement and personnel selection were described.

Monica and Bill All the Time and Everywhere The Collapse of Gatekeeping and Agenda Setting in the New Media Environment

This article argues that by providing virtually unlimited sources of political information, the new media environment undermines the idea that there are discrete gates through which political information passes: If there are no gates, there can be no gatekeepers. The difficulty of elites (political and media both) and academics in understanding the Lewinsky scandal stems from their failure to recognize the increasingly limited ability of journalists to act as gatekeepers. The disjuncture between elite attempts to both control and understand the scandal on one hand and the conclusions the public drew about this political spectacle on other hand speaks to some fundamental changes that have occurred in the role of the press in American society in the late 20th century.

Unchained reaction The collapse of media gatekeeping and the Clinton–Lewinsky scandal

In this article we use the Clinton–Lewinsky scandal to illustrate a fundamental change in the contemporary American media environment: the virtual elimination of the gatekeeping role of the mainstream press. The new media environment, by providing virtually unlimited sources of political information (although these sources do not provide anything like an unlimited number of perspectives), undermines the idea that there are discrete gates through which political information passes: if there are no gates, there can be no gatekeepers. This article is part of a larger project in which we argue that alterations in the media environment have eroded the always uneasy distinction between news and entertainment. Overall, this erosion, one result of which is the collapse of the gatekeeping function, is rapidly undermining the commonsense assumptions used by both elites, citizens and scholars to understand the role of the media in a democratic society.

Fitting Measurement Models to Vocational Interest Data: Are Dominance Models Ideal?

In this study, the authors examined the item response process underlying 3 vocational interest inventories: the Occupational Preference Inventory (C.-P. Deng, P. I. Armstrong, & J. Rounds, 2007), the Interest Profiler (J. Rounds, T. Smith, L. Hubert, P. Lewis, & D. Rivkin, 1999; J. Rounds, C. M. Walker, et al., 1999), and the Interest Finder (J. E. Wall & H. E. Baker, 1997; J. E. Wall, L. L. Wise, & H. E. Baker, 1996). Item response theory (IRT) dominance models, such as the 2-parameter and 3-parameter logistic models, assume that item response functions (IRFs) are monotonically increasing as the latent trait increases. In contrast, IRT ideal point models, such as the generalized graded unfolding model, have IRFs that peak where the latent trait matches the item. Ideal point models are expected to fit better because vocational interest inventories ask about typical behavior, as opposed to requiring maximal performance. Results show that across all 3 interest inventories, the ideal point model provided better descriptions of the response process. The importance of specifying the correct item response model for precise measurement is discussed. In particular, scores computed by a dominance model were shown to be sometimes illogical: individuals endorsing mostly realistic or mostly social items were given similar scores, whereas scores based on an ideal point model were sensitive to which type of items respondents endorsed.

Modeling Incorrect Responses to Multiple-Choice Items with Multilinear Formula Score Theory

Multilinear formula score theory (Levine, 1984, 1985, 1989a, 1989b) provides powerful methods for addressing important psychological measurement prob lems. In this paper, a brief review of multilinear for mula scoring (MFS) is given, with specific emphasis on estimating option characteristic curves (occs). MFS was used to estimate occs for the Arithmetic Reason ing subtest of the Armed Services Vocational Aptitude Battery. A close match was obtained between empiri cal proportions of option selection for examinees in 25 ability intervals and the modeled probabilities of op tion selection. In a second analysis, accurately esti mated occs were obtained for simulated data. To eval uate the utility of modeling incorrect responses to the Arithmetic Reasoning test, the amounts of statistical information about ability were computed for dichoto mous and polychotomous scorings of the items. Con sistent with earlier studies, moderate gains in informa tion were obtained for low to slightly above average abilities.

Measuring the difference between two models

Two psychometric models with very different parametric formulas and item response functions can make virtually the same predictions in all applications. By applying some basic results from the theory of hypothesis testing and from signal detection theory, the power of the most powerful test for distinguishing the models can be com puted. Measuring model misspecification by com puting the power of the most powerful test is proposed. If the power of the most powerful test is low, then the two models will make nearly the same prediction in every application. If the power is high, there will be applications in which the models will make different predictions. This measure, that is, the power of the most powerful test, places various types of model misspecifica tion— item parameter estimation error, multidi mensionality, local independence failure, learning and/or fatigue during testing—on a common scale. The theory supporting the method is presented and illustrated with a systematic study of misspecifica tion due to item response function estimation error. In these studies, two joint maximum likelihood estimation methods (LOGIST 2B and LOGIST 5) and two marginal maximum likelihood estimation methods (BILOG and ForScore) were contrasted by measuring the difference between a simulation model and a model obtained by applying an estimation method to simulation data. Marginal estimation was found generally to be superior to joint estimation. The parametric marginal method (BILOG) was superior to the nonparametric method only for three- parameter logistic models. The nonparametric mar ginal method (ForScore) excelled for more general models. Of the two joint maximum likelihood methods studied, LOGIST s appeared to be more accurate than LOGIST 2B.

Power and sovereignty in hypermedia space: Middle East case studies

For state and non-state actors in the Middle East, the dual imperatives of ontological security and territorial sovereignty has rendered media an important site of contestation, if not conquest. Th...

State Morality in International Relations

Much has recently been said concerning the moral obligations of the state. It is not infrequently asserted that it is the moral duty of a state to assume some function in the interest of international society. The reaction against the philosophy which considered all state action as moral and which posited the realization of national aims as a paramount ethical end, has been followed by an increasing emphasis on the ethical liability of the state to interests in addition to its own. In attributing moral obligations to the state, the ethical standards of the individual are frequently invoked as applicable to state conduct, and upon this analogy judgment is often pronounced on problems of international right and wrong. The question, however, defies settlement by this simple identification of two moral entities essentially dissimilar in their nature. The ascription of ethical duties to the state, wholesome as it is readily conceded to be, requires considerable analysis lest an undue inference be drawn from the mere fact of its admission. To concede the state as a moral entity does not of itself suffice. The manner of its response to moral questions; its distinctive position in a society which yet lacks many of the elemental requisites for moral progress; the forces limiting the movement of international ethics to a higher level—an inquiry into problems such as these would seem more profitable than the constant reiteration of a principle which probably few persons would longer be disposed to deny.