Susan E. Whitely

Multicomponent latent trait models for ability tests

A major research direction for ability measurement has been to identify the information-processes that are involved in solving test items through mathematical modeling of item difficulty. However, this research has had limited impact on ability measurement, since person parameters are not included in the process models. The current paper presents some multicomponent latent trait models for reproducing test performance from both item and person parameters on processing components. Components are identified from item subtasks, in which performance is a logistic function (i.e., Rasch model) of person and item parameters, and then are combined according to a mathematical model of processing on the composite item.

Information Structure for Geometric Analogies: A Test Theory Approach:

Although geometric analogies are popular items for measuring intelligence, the information pro cesses that are involved in their solution have not been studied in a test theory context. In the current study, processing is examined by testing alternative models of information structure on geometric analogies. In contrast to the treatment of models in other studies that have appeared in the cognitive literature, the models are tested jointly as mathe matical models of processing and as latent trait models of individual differences. The joint model ing was achieved by applying the one-parameter lin ear logistic latent trait model to predict response accuracy from information structure. The results supported the model that distinguished between spatial distortion and spatial displacement transfor mations, which have opposite effects on item diffi culty. Further, no significant sex difference in over all accuracy or processing were observed. Implica tions of the results for processing mechanisms and test design are discussed.

Latent Trait Models in the Study of Intelligence.

Abstract This article examines the potential contribution of latent trait models to the study of intelligence. Nontechnical introductions to both unidimensional and multidimensional latent trait models are given, and possible research applications are considered. Latent trait models are shown to resolve several measurement problems in studies of intellectual change, including ability modification studies and life-span development studies. Furthermore, under certain conditions, latent trait models are found useful for construct validation research, since they can represent an individual differences model of cognitive processing on ability test items. Multidimensional latent trait models are shown to be especially useful as processing models, because they can be used to test alternative multiple component theories of test item processing. Furthermore, multidimensional models can be used to decompose test item difficulty into component contributions and estimate individual differences in processing abilities.

Implicit Theories in Student Ratings

Identifying generalizable dimensions of teaching from student ratings data has been both practically and theoretically important in the study of teaching. However, an examination of the methods employed reveals that many studies may have identified dimensions which were implicit in the student raters rather than the instructors. The current study compared various methods of identifying teaching factors and found substantive similarity of the dimensions defined from students’ implicit theories and various types of correlational data. The results are interpreted both with respect to the rating process and to the nature of the factors identified in previous student ratings research.

The Implications of Processing Event Sequences for Theories of Analogical Reasoning

Sternberg (1977a, 1977b) has proposed a componential theory of information processing on analogies. The current study attempts convergent validation of the basic findings in verbal analogies by a method that is based on different underlying assumptions. Although the data were generally consistent with Sternberg’s theory, the data indicated thatapplication is better described by two separate events. Furthermore, the extent of individual differences in strategy models was so substantial that a higher level processing operation, such as control strategies, should be postulated, rather than to support a single-strategy model as characterizing Analogy solving.

Factors Influencing Problem-Solving in Middle-Aged and Elderly Adults.

Groups of middle-aged (30–50 years) and elderly (65–81 years) men and women were compared on three problem-solving tasks, including written problems, the 20-Questions procedure, and problems administe

Validity and generalizability of student ratings from between-classes and within-class data

This article examines the generalizability and validity of student ratings by studying within-class and between-classes correlations of ratings with other variables for regular faculty teaching lecture courses as well as for graduate assistants teaching recitation sections. Results indicate that most ratings are highly generalizable but only some are related to learning and that certain aspects of both generalizability and validity vary with the instructors role and with the level of data. The implications of these findings for the evaluation of teaching are discussed with reference to two alternative paradigms: construct validity and criterion development. Many prior studies have examined aspects of the validity and generalizability of student ratings of instruction. For example, the validity of student ratings as indices of student learning has been addressed by Remmers, Martin, and Elliott (1949), Elliott (1950), Rodin and Rodin (1972), Frey (1973), Sullivan and Skanes (1974), Doyle and Whitely (1974), and Centra (Note 1). Findings have ranged from high negative to high positive correlations, with the majority indicating a statistically significant but very modest relationship between student ratings and tested student learning. Generalizability has been even more frequently studied. DeWolf (Note 2) lists 76 student characteristics that have been examined as possible correlates of student ratings, and Doyle (1975) cites more than 50 generalizabil ity studies. Again the findings are inconsistent, showing sometimes positive, sometimes negative, and sometimes zero correlations between student ratings and various student characteristics. One important explanation of these inconsistencies may be the failure to study a variety of validity and generalizability variables simultaneously and in a single paradigm. Multivariate strategies reduce the risk of focusing too much attention on iso

Information-processing on intelligence test items: Some response components

The reorientation of experimental psychology from studying performance to studying cognitive processes has created a new potential for under standing ability tests in terms of the nature of the cognitive events which contribute to individual dif ferences in solving the test items. The results from the present study suggest the feasibility of explain ing individual differences in performance on a pro totypic intelligence test item—verbal analogies— from the success and efficiency of processing infor mation on hypothesized component events. The data revealed that at least three types of processing events are needed to describe individual differences in the component task durations, but that probably only one factor is needed to describe accuracy in completing the components. More critically, both the accuracy and duration of the component tasks were significantly related to solving psychometric analogies. The results are discussed with respect to the nature of successful performance on analogy test items and the need for more complex models to fully account for individual differences in test per formance.

Individual inconsistency: Implications for test reliability and behavioral predictability

The nature of individual inconsistency in per formance on trait measurements is an important topic in psychometrics because of its direct rele vance to measurement reliability. Several studies have supported short-term inconsistency as a sys tematic source of variation among individuals by finding some evidence for generalizability and rela tionship to behavioral predictability. However, these findings are questionable, since these studies con founded change with short-term fluctuation in their response inconsistency measure. The current re search separates these two sources of inconsistency in a reanalysis of the data from one major study on short-term consistency and finds little evidence for generalizability or a relationship to behavioral pre dictability. These results support the popular as sumption that measurement error from short-term fluctuations is not due to systematic individual dif ferences in response consistency, as well as sup porting a more limited definition of the individual inconsistency construct.

Problem restructuring processes for ill-structured verbal analogies

The processes involved in analogy solving have been an important investigative area in cognitive psychology. Although problem restructuring has been a central construct in problem solving theory, no restructuring processes have been proposed for analogical reasoning. Yet, the stimulus terms for analogies, as they appear on ability tests, are often ill-structured. That is, they are ordered in a way that does not permit direct problem comprehension. In the current study, both perceptual and semantic problem restructuring processes were hypothesized for analogy solving. The independence, stage of execution, and susceptibility to strategic control of the two processes were examined. The results from two experiments indicated that (1) ill-structured analogies are restructured during problem solving, (2) perceptual and semantic restructuring processes are independent and executed at different stages of analogy solving, and (3) both processes exhibited automaticity since repetition of analogy solution attenuated but did not eliminate either restructuring process. A model of analogical reasoning that incorporated both restructuring processes and their execution sequences was proposed. The nature and automaticity of perceptual and semantic analogy restructuring processes were disscussed.