Samuel Messick

Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning.

The traditional conception of validity divides it into three separate and substitutable types—namely, content, criterion, and construct validities. This view is fragmented and incomplete, especially because it fails to take into account both evidence of the value implications of score meaning as a basis for action and the social consequences of score use. The new unified concept of validity interrelates these issues as fundamental aspects of a more comprehensive theory of construct validity that addresses both score meaning and social values in test interpretation and test use. That is, unified validity integrates considerations of content, criteria, and consequences into a construct framework for the empirical testing of rational hypotheses about score meaning and theoretically relevant relationships, including those of an applied and a scientific nature. Six distinguishable aspects of construct validity are highlighted as a means of addressing central issues implicit in the notion of validity as a unified concept. These are content, substantive, structural, generalizability, external, and consequential aspects of construct validity. In effect, these six aspects function as general validity criteria or standards for all educational and psychological measurement, including performance assessments, which are discussed in some detail because of their increasing emphasis in educational and employment settings.

The Interplay of Evidence and Consequences in the Validation of Performance Assessments

Authentic and direct assessments of performances and products are examined in the light of contrasting functions and purposes having implications for validation, especially with respect to the need for specialized validity criteria tailored for performance assessment. These include contrasts between performances and products, between assessment of performance per se and performance assessment of competence or other constructs, between structured and unstructured problems and response modes, and between breadth and depth of domain coverage. These distinctions are elaborated in the context of an overarching contrast between task-driven and construct-driven performance assessment. Rhetoric touting performance assessments because they eschew decomposed skills and decontextualized tasks is viewed as misguided, in that component skills and abstract problems have a legitimate place in pedagogy. Hence, the essence of authentic assessment must be sought elsewhere, that is, in the quest for complete construct representation. With this background, the concepts of “authenticity” and “directness” of performance assessment are treated as tantamount to promissory validity claims that they offset, respectively, the two major threats to construct validity, namely, construct underrepresentation and construct-irrelevant variance. With respect to validation, the salient role of both positive and negative consequences is underscored as well as the need, as in all assessment, for evidence of construct validity.

Meaning and Values in Test Validation: The Science and Ethics of Assessment

Because both score meaning and the value implications of scores as a basis for action are central issues in test validation, a unified view of validity is required that comprehends both the scientific and the ethical underpinnings of test interpretation and use. This unified concept of validity integrates considerations of content, criteria, and consequences into a construct framework for testing rational hypotheses about theoretically relevant relationships, including those of an applied as well as of a scientific nature. The essence of unified validity is that the appropriateness, meaningfulness, and usefulness of score-based inferences are inseparable and that the unifying force behind this integration is the trustworthiness of empirically grounded score interpretation, that is, construct validity.

The nature of cognitive styles: Problems and promise in educational practice

This article examines characteristic features of cognitive styles and the various ways in which styles differ from one another and from intellective abilities. These distinctions are integrated into a unified framework that serves to define cognitive styles in contrast not only to abilities but to other types of stylistic variables. Educational implications of cognitive styles are discussed in terms of six main rubrics: improving instructional methods, enriching teacher behavior and conceptions, enhancing student learning and thinking strategies, expanding guidance and vocational decision making, broadening educational goals and outcomes, and tuning the stylistic demands of educational environments. Reasons why cognitive styles should have educational impact are addressed as well as reasons why such educational benefits are difficult to realize.

Test Validity: A Matter of Consequence

In this note I comment briefly on Keith Markuss illuminating article on “Science, measurement, and validity: Is completion of Samuel Messicks synthesis possible?” Markuss analysis bears directly on the controversial status of the consequential basis of test validity in relation to the more traditional evidential basis. After addressing some key points in his argument, I then comment more generally on sources of the controversy over the claim that empirical consequences of test interpretation and use constitute validity evidence.

Acquiescence and Desirability As Response Determinants On the MMPI

THE aim of the present study was to determine, in a particular population, the relative contributions to response variance on the MMPI of generalized response styles on the one hand and of consistent responses to content on the other, with primary emphasis given to the stylistic tendencies to agree and to respond desirably. One specific aim of this research was to evaluate the hypothesis (Jackson & Messick, 1958) that consistent tendencies to acquiesce will be elicited differentially by items at varying levels of desirability. Beyond appraising the effects of desirability and acquiescence, however, a further aim of the study was to evaluate possible consistencies attributable to specific item content in the present form of the MMPI, after first partialing out statistically the variance traceable to stylistic response consistencies. We also wished to obtain some preliminary information about several possible alternatives in the use of structured personality

Evaluation of Group and Individual Forms of Embedded-Figures Measures of Field-Independence 1

(1950) Embedded-Figures Test (EFT) ; second, to study the roles of various response determinants, including the influence of memory, color, and item format upon individual consistencies in the extraction of embedded figures; and third, to appraise relationships between performance on group and individual embedded-figures tests and selected cognitive and personality measures. Interest in studying perceptual field-independence and its ramifications in personality theory, development, and cognition has grown steadily since the appearance of a report describing a decade of research by Witkin and his associates (Witkin et al., 1954). At

Potential Uses of Noncognitive Measurement in Education.

The potential educational import of noncognitive personal characteristics is examined with particular reference to problems in their measurement. Several noncognitive variables are discussed in terms of their distinctive features and educational relevance, including experiential background factors, affects, attitudes, interests, motives, curiosity, temperament, social sensitivity, coping strategies, cognitive styles, creativity, and values. Their different roles in various educational functions are briefly examined, especially in improving access and educational opportunity, setting objectives and standards, guidance, selection, placement, instruction and learning, and evaluation. Finally, attention is drawn to some dangers and difficulties in using noncognitive measures in educational practice and to the need for safeguards against misuse.

Desirability Scale Values and Dispersions for Mmpi Items

In the course of a large research project investigating stylistic influences in responses to personality inventories (cf. Jackson & Messick, 1960), ratings of the judged desirability of all items appearing in the booklet form of the Minnesota Multiphasic Personality Inventory (MMPI) were collected from a sample of college students. These desirability ratings were obtained primarily to ascertain the variety of differential viewpoints that might be represented among normal college students in judging the desirability of personality and pathological characteristics reflected in MMPI item content (cf. Messick, 1960). Although the primary concern of the study was thus to uncover consistent individual differences in the ratings of desirability, it was also felt that some measure of average judged desirability and of the variability of the ratings for each item would be useful for future scale development and evaluation. Such average item desirability values might be used, for example, in selecting items to measure consistent tendencies to respond desirably, in attempts to counterbalance item desirability in constructing content scales, or in selecting items of presumably matched desirability for forced-choice formats (Edwards, 1957 ) . The variability or dispersion indices could also serve as an additional criterion for item matching. In the present study, the scaling method of successive intervals was used to obtain a desirability scale value and discriminal dispersion for each MMPI item (cf. Torgerson, 1958). This particular scaling technique attempts to obtain comparability of ratings across items, not by assuming that rating categories are equally spaced as in the method of equal-appearing intervals, but by adjusting the interval widths so as to normalize simultaneously all of the rating distributions on the same base line. The scale values derived by this method essentially represent the means of these normalized item distributions on the common scale, and the discriminal dispersions are the corresponding standard deviations. If the scaling model fits the data, interval measurement is achieved (Adams & Messick, 1958), and differences in the resulting average desirability scale values are directly comparable.

Assessment in Context: Appraising Student Performance in Relation to Instructional Quality

The following articles are modified from a symposium organized by Jeremy Finn and presented at the 1983 Annual Meeting of the American Educational Research Association in Montreal, Quebec. The papers by Samuel Messick and by Jeremy Finn and Lauren Resnick summarize the major findings and implications of the National Academy of Sciences investigation; critiques of these presentations by Daniel Reschly and Richard Snow follow. Dr. Messicks paper begins with an overview of the problem as originally formulated for the panel and the procedures followed in the investigation.