Samuel A. Livingston

A Comparative Study of Standard-Setting Methods

The borderline-group method and the contrasting-groups method were compared with Nedelskys method at four schools, with Angoffs method at another four schools, and with each other at all eight schools, using tests of basic skills in reading and mathematics. The borderline-group and contrastinggroups methods produced similar results when approximately equal numbers of students were classified as masters and nonmasters. The contrasting-groups passing score was lower than the borderline-group passing score when masters greatly outnumbered nonmasters and higher when nonmasters outnumbered masters. Results involving the Nedelsky and Angoff methods were not consistent across schools. Passing scores tended to be higher at schools where students were more able.

Generalized Equating Functions for NEAT Designs

The purpose of this chapter is to introduce generalized equating functions for the equating of test scores through an anchor. Depending on the choice of parameter values, the generalized equating function can perform either linear equating or equipercentile equating, either by poststratification on the anchor or by chained linking through the anchor.

Collateral Information for Equating in Small Samples: A Preliminary Investigation

This article describes a preliminary investigation of an empirical Bayes (EB) procedure for using collateral information to improve equating of scores on test forms taken by small numbers of examinees. Resampling studies were done on two different forms of the same test. In each study, EB and non-EB versions of two equating methods—chained linear and chained mean—were applied to repeated small samples drawn from a large data set collected for a common-item equating. The criterion equating was the chained linear equating in the large data set. Equatings of other forms of the same test provided the collateral information. New-form sample size was varied from 10 to 200; reference-form sample size was constant at 200. One of the two new forms did not differ greatly in difficulty from its reference form, as was the case for the equatings used as collateral information. For this form, the EB procedure improved the accuracy of equating with new-form samples of 50 or fewer. The other new form was much more difficult than its reference form; for this form, the EB procedure made the equating less accurate.

A Case of Inconsistent Equatings: How the Man With Four Watches Decides What Time It Is

A simultaneous equating of four new test forms to each other and to one previous form was accomplished through a complex design incorporating seven separate equating links. Each new form was linked to the reference form by four different paths, and each path produced a different score conversion. The procedure used to resolve these inconsistencies was applied separately at each score level. Considering each equating (at a given score level) as a simple additive increment and imposing constraints on those increments led to a system of seven equations in seven unknowns. The solution produced a set of adjusted increments, so that the linking of any new form to the reference form was the same by all four possible paths.

New Approaches to Equating With Small Samples

The purpose of this chapter is to introduce the reader to some recent innovations intended to solve the problem of equating test scores on the basis of data from small numbers of test takers. We begin with a brief description of the problem and of the techniques that psychometricians now use in attempting to deal with it. We then describe three new approaches to the problem, each dealing with a different stage of the equating process: (1) data collection, (2) estimating the equating relationship from the data collected, and (3) using collateral information to improve the estimate. We begin with Stage 2, describing a new method of estimating the equating transformation from small-sample data. We also describe the type of research studies we are using to evaluate the effectiveness of this new method. Then we move to Stage 3, describing some procedures for using collateral information from other equatings to improve the accuracy of an equating based on small-sample data. Finally, we turn to Stage 1, describing a new data collection plan in which the new form is introduced in a series of stages rather than all at once.