Frank B. Baker

Item response theory : parameter estimation techniques

The Item Characteristic Curve: Dichotomous Response Estimating the Parameters of an Item Characteristic Curve Maximum Likelihood Estimation of Examinee Ability Maximum Likelihood Procedures for Estimating Both Ability and Item Parameters The Rasch Model Marginal Maximum Likelihood Estimation and an EM Algorithm Bayesian Parameter Estimation Procedures The Graded Item Response Nominally Scored Items Markov Chain Monte Carlo Methods Parameter Estimation with Multiple Groups Parameter Estimation for a Test with Mixed Item Types

Measuring the Power of Hierarchical Cluster Analysis

Abstract The concept of power for monotone invariant clustering procedures is developed via the possible partitions of objects at each iteration level in the obtained hierarchy. At a given level, the probability of rejecting the randomness hypothesis is obtained empirically for the possible types of partitions of the n objects employed. The results indicate that the power of a particular hierarchical clustering procedure is a function of the type of partition. The additional problem of estimating a “true” partition at a certain level of a hierarchy is discussed briefly.

The Analysis of Social Interaction Data

A nonparametric technique is presented that is appropriate for comparing two social interaction matrices, either when both are obtained empirically, or when one is generated from some given theoretical position. Depending on whether the diagonals of the matrices are considered relevant to the analysis, the com parison can be carried out using some variation on a cross-product statistic. In all cases, the chosen index is referred to an exact (or approrimate) permutation dis tribution, based on the random matching of the rows and columns of one matrix to those of a second. Several examples are provided along with formulas for the first two moments of the permutation distributions for the suggested indices.

Detection of differential item functioning in the graded response model

Methods for detecting differential item func tioning (DIF) have been proposed primarily for the item response theory dichotomous response model. Three measures of DIF for the dichotomous response model are extended to include Samejimas graded response model: two measures based on area differences between item true score functions, and a χ2 statistic for comparing differences in item parameters. An illustrative example is presented.

Stability of Two Hierarchical Grouping Techniques Case I: Sensitivity to Data Errors

Abstract The impact of observational errors on the dendrograms produced by the complete linkage and single linkage hierarchical grouping techniques was investigated. Data corresponding to basal taxonomies were perturbed by adding error to the ranks of the inter-specimen pair proximities. The goodness of fit of the dendrograms obtained from the perturbed data and that corresponding to the basal taxonomies were measured by means of the Goodman-Kruskal gamma coefficient. Empirical sampling distributions of the gamma coefficients indicated that the single linkage grouping technique was more sensitive to the type of data errors employed than the complete linkage technique.

Equating tests under the graded response model

The Stocking and Lord (1983) procedure for computing equating coefficients for tests having dichotomously scored items is extended to the case of graded response items. A system of equations for obtaining the equating coefficients under Samejimas (1969, 1972) graded response model is derived. These equations are used to compute equating coefficients in two related situations. Under the first, the equating coefficients are obtained by matching, on an examinee by examinee basis, the true scores on two tests. In the second case, the equating coefficients are obtained by matching the test characteristic curves (TCCS) of the two tests. Several examples of computing equating coefficients in these two situations are provided. The TCC matching ap proach was much less demanding computationally and yielded equating coefficients that differed little from those obtained through the true score distribution matching approach.

Evaluating the Conformity of Sociometric Measurements.

The problem of comparing two sociometric matrices, as originally discussed by Katz and Powell in the early 1950s, is reconsidered and generalized using a different inference model. In particular, the proposed indices of conformity are justified by a regression argument similar to the one used by Somers in presenting his well-known measures of asymmetric ordinal association. A permutation distribution and an associated significance test are developed for the specific hypothesis of “no conformity” reinterpreted as a random matching of the rows and (simultaneously) the columns of one sociometric matrix to the rows and columns of a second. The approximate significance tests that are presented and illustrated with a simple numerical example are based on the first two moments of the permutation distribution, or alternatively, on a random sample from the complete distribution.

An Investigation of the Item Parameter Recovery Characteristics of a Gibbs Sampling Procedure.

The item parameter recovery characteristics of a Gibbs sampling method (Albert, 1992) for IRT item parameter estimation were investigated using a simulation study. The item parameters were estimated, under a normal ogive item response function model, using Gibbs sampling and BILOG (Mislevy & Bock, 1989). The item parameter estimates were then equated to the metric of the underlying item parameters for tests with 10, 20, 30, and 50 items, and samples of 30, 60, 120, and 500 examinees. Summary statistics of the equating coefficients showed that Gibbs sampling and BILOG both produced trait scale metrics with units of measurement that were too small, but yielding a proper midpoint of the metric. When expressed in a common metric, the biases of the BILOG estimates of the item discriminations were uniformly smaller and less variable than those from Gibbs sampling. The biases of the item difficulty estimates yielded by the two estimation procedures were small and similar to each other. In addition, the item parameter recovery characteristics were comparable for the largest dataset of 50 items and 500 examinees. However, for short tests and sample sizes the item parameter recovery characteristics of BILOG were superior to those of the Gibbs sampling approach.

A Graph-Theoretic Approach to Goodness-of-Fit in Complete-Link Hierarchical Clustering

Abstract The complete-link hierarchical clustering strategy is reinterpreted as a heuristic procedure for coloring the nodes of a graph. Using this framework, the problem of assessing goodness-of-fit in complete-link clustering is approached through the number of “extraneous” edges in the fit of the constructed partitions to a sequence of graphs obtained from the basic proximity data. Several simple numerical examples that illustrate the suggested paradigm are given and some Monte Carlo results presented.

The comparison and fitting of given classification schemes

Abstract A permutation procedure is described for statistically comparing a given classification scheme, characterized as a hierarchically organized collection of subsets, to either a proximity matrix or a second classification scheme defined on the same basic set of objects. To prevent a bias that may result when an optimization method is used to construct the classification structures being considered, it is assumed that the desired comparisons involve classifications and/or proximities that have been derived from separate sources. In addition to an extensive number of examples used to clarify the major points in the presentation of the inference strategy, several heuristic optimization techniques are also introduced and illustrated. These latter procedures attempt to “fit” the form of a target classification structure by relabeling the rows and simultaneously the columns of a given proximity matrix to maximize the correspondence between the fixed target and the relabeled proximity matrix.