Michael D. Maraun

Measurement as a Normative Practice Implications of Wittgenstein's Philosophy for Measurement in Psychology

Recently, a number of prominent measurement specialists (e.g. Cliff, 1992; Schonemann, 1994) have pondered the lack of progress in the development of convincing solutions to the measurement problems of psychology, and have attempted to identify the factors responsible for this lack of progress. They suggest a number of possibilities, including a basic lack of talent in the ranks of the social sciences. It is argued here, however, that the philosophy of Wittgenstein provides an interesting alternative explanation. Specifically, despite their apparent differences, current approaches to the support of psychological measurement claims are unanimous in viewing measurement as chiefly an empirical matter. On Wittgensteins account, however, this is a mischaracterization of measurement, for, as he argued in elaborate detail, measurement is a normative, rule-guided practice. Hence, empirically based argument is not relevant to the support of measurement claims. If this verdict is correct, it explains not only the failure of measurement theory in psychology, but the much discussed success of measurement in the physical sciences. In this paper, Wittgensteins characterization of measurement, and its implications for psychology, are discussed.

Appearance and reality: Is the Big Five the structure of trait descriptors?

Abstract It is argued that contrary to the claims of Big Five investigators, the structure of trait descriptors is still very much an open issue. This is because their methodology, factor/component analysis paired with the dimensional interpretation/simple structure (DISS) procedure, does not investigate the closed topological manifold that constitutes the ‘structure’ of a set of variables. Instead, radex-related configurations are likely candidates for the structure of trait descriptors. Some preliminary support for this claim is given by an analysis of the NEO Personality Inventory (NEO-PI) (Costa & McCrae, Manual for the NEO PI-R . Odessa, FL: Psychological Assessment Resources, 1992), a Big Five questionnaire measure, and the Goldberg-40 (Goldberg, Review of Personality and Social Psychology , Vol. 2, pp. 141–165, 1981. Beverly Hills, CA: Sage), an adjective measure. In particular, the NEO-PI and Goldberg-40 are shown to have radex structures. A facet theory (Guttman, Psychometrika, 36 , 329–346, 1971) rationale is provided for these findings.

The conceptual validity of empirical scale construction: The case of the sensation seeking scale

Abstract An assessment of the present state of sensation seeking (SS) research suggests that the concepts of physical risk taking and sensation seeking have become synonymous with version five of Zuckermans Sensation Seeking Scale (SSS). Given this, it would appear that the validity of SS research rests on the extent to which the SSS actually measures risk taking and sensation seeking tendencies. The present paper suggests that a series of logical and conceptual errors in the development of the SSS have resulted in a scale that does not measure sensation seeking. This raises concerns about both the value and meaning of investigations in which it is employed. It is concluded that risk taking and SS research has learned more about the SSS than sensation seeking itself.

A proposed framework for conducting data-based test analysis.

The authors argue that the current state of applied data-based test analytic practice is unstructured and unmethodical due in large part to the fact that there is no clearly specified, widely accepted test analytic framework for judging the performances of particular tests in particular contexts. Drawing from the extant test theory literature, they propose a rationale that may be used in data-based test analysis. The components of the proposed test analytic framework are outlined in detail, as are examples of the framework as applied to commonly encountered test evaluative scenarios. A number of potential extensions of the framework are discussed.

An Analysis of Meehl's MAXCOV-HITMAX Procedure for the Case of Dichotomous Indicators

MAXCOV-HITMAX was invented by Paul Meehl for the detection of latent taxonic structures (i.e., structures in which the latent variable, _, is not continuously, but rather Bernoulli, distributed). It involves the examination of the shape of a certain conditional covariance function, and is based on Meehls claims that: (R1) Given a latent taxonic structure, this conditional covariance function is single peaked; and that (R2), continuous latent structures produce a flat, rather than single-peaked, curve. While Meehl has recommended that continuous indicators be used as input into MAXCOV-HITMAX, the use of dichotomous indicators has become popular. The current work investigates whether (R1) and (R2) are true for the case of dichotomous indicators. The conclusions will be that, for dichotomous indicators: (a) (R1) is not true; (b) (R1) is made true given that there are a large number of indicators; and (c) (R2) is not true, certain unexceptional Rasch structures, for example, producing single-peaked curves. Implications are briefly discussed of these results for the case of MAXCOV-HITMAX with continuous indicators.

Psychometric Assessment and Reporting Practices: Incongruence between Theory and Practice.

The aim of the current study is twofold: (a) to investigate the rates at which researchers assess and report on the psychometric properties of the measures they use in their research and (b) to examine whether or not researchers appear to be generally employing sound/unsound rationales when it comes to how they conduct test evaluations. Based on a sample of 368 articles published in four journals in the year 2004, the findings suggest that, although evidence bearing on score precision/reliability and the internal structure of item responses remains under-reported, researchers appear to be assessing the relationships between test scores and external variables relatively more frequently than in the past. However, findings also indicate that, all told, very few researchers are assessing and reporting on internal score validity, and score precision/reliability, and external score validity, and in that sequence, suggesting that applied researchers may not always be adopting sound test-evaluative rationales in their psychometric assessments.

Manifest and Latent Variates

Michell, J. (2000). Normal science, pathological science and psychometrics. Theory & Psychology, 10, 639–667. Michell, J. (2004). Item response models, pathological science and the shape of error: Reply to Borsboom and Mellenbergh. Theory & Psychology, 14, 121–129. Michell, J. (2008). Conjoint measurement and the Rasch paradox: A response to Kyngdon. Theory & Psychology, 18, 119–124. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research. Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372–422. Rayner, K., Reichle, E. D., Stroud, M. J., Williams, C. C., & Pollatsek, A. (2006). The effect of word frequency, word predictability and font difficulty on the eye movements of young and older readers. Psychology and Aging, 21, 448–465. Shankweiler, D., & Crain, S.(1986). Language mechanisms and reading disorder: A modular approach. Cognition, 14, 139–168. Stenner, A. J., Burdick, H., Sanford, E. E., & Burdick, D. S. (2006). How accurate are Lexile text measures? Journal of Applied Measurement, 7, 307–322. Stenner, A. J., Smith, M., & Burdick, D. S. (1983). Toward a theory of construct definition. Journal of Education Measurement, 20, 305–315. von Eye, A. (2005). Review of Cliff and Keats, Ordinal measurement in the behavioral sciences. Applied Psychological Measurement, 29, 401–403.

The mythologization of regression towards the mean

In the quantitative methodology literature, there now exists what can be considered a received account of the enigmatic phenomenon known as regression towards the mean (RTM), the origins of which can be traced to the work of Sir Francis Galton circa 1885. On the received account, RTM is, variably, portrayed as a ubiquitous, unobservable property of individual-level difference and change phenomena, a force that impacts upon the characteristics of individual entities, an explanation for difference and change phenomena, and a profound threat to the drawing of correct conclusions in experiments. In the current paper, we describe the most essential components of the received account, and offer arguments to the effect that the received account is a mythologization of RTM. In particular, we: (a) describe the scientific and statistical setting in which a consideration of RTM is embedded; (b) translate Galton’s discussion of RTM into modern statistical terms; (c) excavate a definition of the concept regression towards the mean from Galton’s discussion of RTM; and (d) employ the excavated definition to dismantle certain of the most essential components of the received account.

The Extra-Factor Phenomenon Revisited: Unidimensional Unfolding as Quadratic Factor Analysis.

The application of linear factor analysis to a set of unfoldable (unidimensional) items produces a two-dimensional solution, called the extra-factor phenomenon, which potentially results in incorrect conclusions about the nature of a set of items (van Schuur& Kiers, 1994). Many explanations have been offered for this phenomenon. This study attempted further clarification within the general theory of factor analysis. Specifically, it was demonstrated that the extra-factor phenomenon arises because: (1) the metric unidimensional unfolding model is equivalent to the unidimensional quadratic factor model; and (2) at the level of covariance structure, the unidimensional quadratic factor model is not distinguishable from the two-dimensional linear factor model (McDonald, 1967). Also discussed are a number of theoretical linkages and bases of distinguishability that exist between unidimensional unfolding and linear factor analysis.

Validity and Measurement.

As illuminated forcefully by Professor Newtons provocative analytical and historical excursion, as long as tests are employed to practical ends (prediction, selection, etc.) there is little cause ...