Robert L. Brennan

Practical Issues in Equating

Many of the practical issues that are involved in conducting equating are described in this chapter. We describe major issues and provide references that consider these issues in more depth. The early portions of this chapter focus on equating dichotomously scored paper-and-pencil tests. In later portions, the focus broadens to include practical issues in other contexts, including computerized testing and performance assessments. Various articles have been written that review practical issues in equating (Brennan and Kolen, 1987b; Cook and Petersen, 1987; Harris, 1993; Harris and Crouse, 1993; Skaggs, 1990a; and Skaggs and Lissitz, 1986b) in greater depth than those provided in this chapter.

Nonequivalent Groups-Linear Methods

Chapter 1 introduced the common-item nonequivalent groups design. For this design, two groups of examinees from different populations are each administered different test forms that have a set of items in common This design often is used when only one form of a test can be administered on a given test date. As discussed in Chapter 1, the set of common items should be as similar as possible to the full-length forms in both content and statistical characteristics.

Standard Errors of Equating

Two general sources of error in estimating equating relationships are present whenever equating is conducted using data from an equating study: random error and systematic error. Random equating error is present when the scores of examinees who are considered to be samples from a population or populations of examinees are used to estimate equating relationships. When only random equating error is involved in estimating equating relationships, the estimated equating relationship differs from the equating relationship in the population because data were collected from a sample, rather than from the whole population. If the whole population were available, then no random equating error would be present. Thus, the amount of random error in estimating equating relationships becomes negligible as the sample size increases.

Item Response Theory Methods

In this chapter, we describe item response theory (IRT) equating methods under various designs. This chapter covers issues that include scaling person and item parameters, IRT true and observed score equating methods, equating using item pools, and equating using polytomous IRT models.

Observed Score Equating Using the Random Groups Design

As was stressed in Chapter 1, the same specifications property is an essential property of equating, which means that the forms to be equated must be built to the same content and statistical specifications. We also stressed that the symmetry property is essential for any equating relationship. The focus of the present chapter is on methods that are designed to achieve the observed score equating property, along with the same specifications and symmetry properties. As was described in Chapter 1, these observed score equating methods are developed with the goal that, after equating, converted scores on two forms have at least some of the same score distribution characteristics in a population of examinees.

Nonequivalent Groups: Equipercentile Methods

Equipercentile equating methods have been developed for the common-item nonequivalent groups design. These methods are similar to the equipercentile methods for random groups described in Chapter 2. Equipercentile methods with nonequivalent groups consider the distributions of total score and scores on the common items, rather than only the means, standard deviations, and covariances that were considered in Chapter 4. As has been indicated previously, equipercentile equating is an observed score equating procedure that is developed from the perspective of the observed score equating property described in Chapter 1. Thus, equipercentile equating with the common-item nonequivalent groups design requires that a synthetic population, as defined in Chapter 4, be considered. In this chapter, we present an equipercentile method that we show to be closely allied to the Tucker linear method of Chapter 4. We also describe how smoothing methods, such as those described in Chapter 3, can be used when conducting equipercentile equating with nonequivalent groups. The methods described in this chapter are illustrated using the same data that were used in Chapter 4, and the results are compared to the linear results from Chapter 4.

Random Groups-Smoothing in Equipercentile Equating

As described in Chapter 2, sample statistics are used to estimate equating relationships. For mean and linear equating, the use of sample means and standard deviations in place of the parameters typically leads to adequate equating precision, even when the sample size is fairly small. However, when sample percentiles and percentile ranks are used to estimate equipercentile relationships, equating often is not sufficiently precise for practical purposes because of sampling error.

Current and Future Challenges

In Chapter 1, we summarized the concepts of equating, scaling, and linking In subsequent chapters, these concepts were further developed and elaborated. In Chapter 11 we focus on some current and future challenges in each of these areas.