W. Alan Nicewander

Thirteen Ways to Look at the Correlation Coefficient

Abstract In 1885, Sir Francis Galton first defined the term “regression” and completed the theory of bivariate correlation. A decade later, Karl Pearson developed the index that we still use to measure correlation, Pearsons r. Our article is written in recognition of the 100th anniversary of Galtons first discussion of regression and correlation. We begin with a brief history. Then we present 13 different formulas, each of which represents a different computational and conceptual definition of r. Each formula suggests a different way of thinking about this index, from algebraic, geometric, and trigonometric settings. We show that Pearsons r (or simple functions of r) may variously be thought of as a special type of mean, a special type of variance, the ratio of two means, the ratio of two variances, the slope of a line, the cosine of an angle, and the tangent to an ellipse, and may be looked at from several other interesting perspectives.

Some symmetric, invariant measures of multivariate association

A distinction is drawn between redundancy measurement and the measurement of multivariate association for two sets of variables. Several measures of multivariate association between two sets of variables are examined. It is shown that all of these measures are generalizations of the (univariate) squared-multiple correlation; all are functions of the canonical correlations, and all are invariant under linear transformations of the original sets of variables. It is further shown that the measures can be considered to be symmetric and are strictly ordered for any two sets of observed variables. It is suggested that measures of multivariate relationship may be used to generalize the concept of test reliability to the case of vector random variables.

Ability estimation for conventional tests

Five different ability estimators—maximum likelihood [MLE (θ)], weighted likelihood [WLE (θ)], Bayesian modal [BME (θ)], expected a posteriori [EAP (θ)] and the standardized number-right score [Z (θ)]—were used as scores for conventional, multiple-choice tests. The bias, standard error and reliability of the five ability estimators were evaluated using Monte Carlo estimates of the unknown conditional means and variances of the estimators. The results indicated that ability estimates based on BME (θ), EAP (θ) or WLE (θ) were reasonably unbiased for the range of abilities corresponding to the difficulty of a test, and that their standard errors were relatively small. Also, they were as reliable as the old standby—the number-right score.

Linearly Independent, Orthogonal, and Uncorrelated Variables

Abstract Linearly independent, orthogonal, and uncorrelated are three terms used to indicate lack of relationship between variables. This short didactic article compares these three terms in both an algebraic and a geometric framework. An example is used to illustrate the differences.

Some Reliability Estimates for Computerized Adaptive Tests

Three reliability estimates are derived for the Bayes modal estimate (BME) and the maximum likelihood estimate (MLE) of θin computerized adaptive tests (CAT). Each reliability estimate is a function of test information. Two of the estimates are shown to be upper bounds to true reliability. The three reliability estimates and the true reliabilities of both MLE and BME were computed for seven simulated CATs. Results showed that the true reliabilities for MLE and BME were nearly identical in all seven tests. The three reliability estimates never differed from the true reliabilities by more than .02 (.01 in most cases). A simple implementation of one reliability estimate was found to accurately estimate reliability in CATs.

Alternative Response and Scoring Methods for Multiple Choice Items: An Empirical Study of Probabilistic and Ordinal Response Modes.

Binary, probability, and ordinal scoring proce dures for multiple-choice items were examined. In a situation where true scores were experimentally controlled by the manipulation of partial informa tion, it was found that both the probability and or dinal scoring systems were more reliable than the binary scoring method. A second experiment using vocabulary items and standard reliability estimation procedures also showed higher reliability for the two partial information scoring methods relative to binary scoring.

A Comparison of Paper and Online Tests Using a within-Subjects Design and Propensity Score Matching Study

This inquiry had 2 components: (1) the first was substantive and focused on the comparability of paper-based and computer-based test forms and (2) the second was a within-study comparison wherein a quasi-experimental method, propensity score matching, was compared with a credible benchmark method, a within-subjects design. The tests used in the comparison of online tests and paper-based tests were End-of-Course tests in Algebra and English, in a statewide high school testing program. Students were tested in Grades 8 and 9. In general, the substantive studies suggested that the online and paper tests appeared to be measuring the same underlying constructs with the same level of reliability. The within-study portion of the investigation indicated that propensity score matching study yielded results that were virtually identical to the outcome of the more conventional within-subjects experimental design. Both the methodological and substantive aspects of this investigation yielded outcomes that should be of interest to investigators in both of these areas.

A relationship between harris factors and guttman's sixth lower bound to reliability

Using an approach nearly identical to one adopted by Guttman, it is established that within the framework of classical test theory the squared multiple correlation for predicting an element of a composite measure from then — 1 remaining elements is a lower-bound to the reliability of the element. The relationship existing between the reliabilities of the elements of a composite measure and their squared-multiple correlations with remaining elements is used to derive Guttmans sixth lower bound (λ6) to the reliability of a composite measure. It is shown that Harris factors of a correlation matrixR are associated with a set of (observable) uncorrelated latent variables having maximum coefficientsλ6.

Model-Based Versus Empirical Equating of Test Forms

Model-based equating was compared to empirical equating of an Armed Services Vocational Aptitude Battery (ASVAB) test form. The model-based equating was done using item pretest data to derive item response theory (IRT) item parameter estimates for those items that were retained in the final version of the test. The analysis of an ASVAB test form indicated that the model-based equatings were nearly as accurate as the preliminary empirical equating, using the final empirical equating as a standard against which to judge the equatings. The fact that the model-based equatings so closely forecast the final ones suggests some indirect support for the assumptions commonly made in IRT analyses—that IRT models reasonably approximate human behavior when confronted by multiple-choice test items and a normal distribution for the latent trait being measured.

A latent-trait based reliability estimate and upper bound

An estimate and an upper-bound estimate for the reliability of a test composed of binary items is derived from the multidimensional latent trait theory proposed by Bock and Aitkin (1981). The estimate derived here is similar to internal consistency estimates (such as coefficient alpha) in that it is a function of the correlations among test items; however, it is not a lowerbound estimate as are all other similar methods.An upper bound to reliability that is less than unity does not exist in the context of classical test theory. The richer theoretical background provided by Bock and Aitkins latent trait model has allowed the development of an index (called δ here) that is always greater-than or equal-to the reliability coefficient for a test (and is less-than or equal-to one). The upper bound estimate of reliability has practical uses—one of which makes use of the “greatest lower bound”.