Qing Yi

Invariance of Equating Functions Across Different Subgroups of Examinees Taking a Science Achievement Test

This study investigated the group invariance of equating results using a science achievement test. Examinees were divided into different subgroups based on the average composite score for test centers, whether they had taken a physics course, and self-reported science grade point average. The reason for dividing examinees into subgroups using such variables is that those variables are more related to performance on a science achievement test than, say, gender. Results indicated that the conversions obtained from different subgroups were similar to the conversions obtained by using the total group, except when the groups were divided based on whether a student had taken a physics course. Where there were differences, the differences were generally 1 equated raw score point.

Mode Comparability Studies for a High-Stakes Testing Program

Mode comparability between paper and online versions of a test cannot be simply assumed. This paper presents the research designs, statistical analyses, and major findings from a series of special studies intended to ensure score comparability for a high-stakes testing program, including an online timing study and two mode comparability studies as well as a general framework that guided the design of these studies. The framework views score comparability as a matter of degree and the evaluation of score comparability as a matter of score validation. The high-stakes uses of the test scores required stringent score comparability which was obtained by applying test-equating methodologies under a randomly equivalent groups design. Meanwhile, score equivalency and construct equivalency were examined through statistical analyses of test results and responses to survey questions. The comparability framework and results from these studies may provide guidance for other testing programs transitioning from paper to online or when evaluating score comparability in general.