Robert A. Forsyth

An Examination of the Characteristics of Unidimensional IRT Parameter Estimates Derived From Two-Dimensional Data

The purpose of this investigation was to study the nature of the item and ability estimates obtained when the modified three-parameter logistic model is used with two-dimensional data. To examine the effects of two-dimensional data on unidimensional parameter es timates, the relative potency of the two dimensions was systematically varied by changing the correlations between the two ability dimensions. Data sets based on correlations of .0, .3, .6, .9, and .95 were gener ated for each of four combinations of sample size and test length. Also, for each of these four combinations, five unidimensional data sets were simulated for com parison purposes. Relative to the nature of the unidi mensional estimates, it was found that the â value seemed best considered as the average of the true a values. The b value seemed best thought of as an overestimate of the true b 1 values. The θ value seemed best considered as the average of the true ability pa rameters. Although there was a consistent trend for these relationships to strengthen as the ability dimen sions became more highly correlated, there was al ways a substantial disparity between the magnitudes of these values and of those derived from the unidimen sional data. Sample size and test length had very little effect on these relationships.

The Comparative Effects of Compensatory and Noncompensatory Two-Dimensional Data on Unidimensional IRT Estimates

This study compared the effects of using a unidi mensional IRT model with two-dimensional data gener ated by noncompensatory and compensatory multidi mensional IRT models. Within each model, simulated datasets differed according to the degree of correlation between two vectors of θ parameters, ranging from 0 to .95. Results showed that the number-correct distri butions for each group of datasets were generally comparable, although factor analyses of tetrachoric correlations suggested that differences existed in the structure of the data from the two models. For the uni dimensional parameter estimates, it was found that the â values from the noncompensatory model appeared to be averages of the a 1 and a 2 values, while the â values from the compensatory model were best considered as an estimate of the sum of the a 1 and a 2 values. Con versely, the b values for the noncompensatory data were consistently greater than the b 1 values, while the b values from the compensatory model were best con sidered as the average of the b 1 and b 2 values. For both models the θ estimates were most highly related to the average of the two θ parameters. However, for the noncompensatory model there was a general in crease in the strength of this relationship with in creases in ρ(θ1,θ 2). For the compensatory model, the strength of this relationship did not show a great deal of change with differences in ρ(θ1,θ 2). Index terms: Compensatory multidimensional IRT models, Item response theory, Multidimensional IRT models, Noncompensatory multidimensional IRT models, Pa rameter estimation, Violations of unidimensionality.

An Empirical Note on Correlation Coefficients Corrected for Restriction in Range.

BEHAVosiAL scientists frequently utilize interval estimation techniques and point estimates of parameters for which there exists only relatively inadequate sampling theories. Correlation coefficients corrected for attenuation, for example, have estimated standard error formulas, but the form of the sampling distribution is not known. Investigators using such coefficients often employ the sampling theory for the Pearson product-moment correlation coefficient (Yamamoto, 1965). The inadequacies associated with this procedure have been discussed recently by Forsyth and Feldt (1969). A fairly similar situation prevails when an experimenter wishes to estimate the correlation between variables X and Y for some population but complete data are available for only a select, nonrandom sample of subjects. In a common example, subjects have been selected on the basis of their X-scores and data for the Y-variable are available only for this restricted group. This situation would occur if a college entrance test was utilized as a selection device and it was desired to estimate the correlation between the test and freshman grade point average for a population of students represented by all those who applied for admission. In this case, data on the test are available for both groups, but grade point data

Construction and Validation of an Instrument Measuring Certain Attitudes toward Mathematics

The purpose of this study was to develop scales measuring four facets of attitude toward mathematics and to assess the reliability and validity of these scales. The four constructs of interest were: enjoyment of word problems; enjoyment of pictorial problems; appreciation of the utility of mathematics; and security with mathematics. The scales designed to measure these four constructs were administered to 299 seventh grade students. Split-half reliability coefficients were estimated for each scale and evidence related to the construct validity of each scale was obtained. All scales demonstrated an adequate degree of reliability and some degree of both convergent and discriminant validity.

Relationship of Elementary and Secondary School Achievement Test Scores To Later Academic Success

This study investigated the relationship between achievement test scores on the ITBS and ITED and academic success (high school and college grade point average). The basic population consisted of approximately 2000 students who entered the University of Iowa as freshmen in 1977. Scores on high school achievement tests adminis tered in grades 9, 10, 11, and 12 and on elementary achievement tests administered in grades 4, 6, and 8 were obtained for a sub stantial portion of this population, along with high school and col lege freshman grade point averages (GPAs). Correlations between these achievement test scores and GPAs were similar to those re ported for tests of the college entrance type. This study provides support for the predictive validity of the ITBS and ITED achieve ment test batteries.

Approximating Standardized Achievement Test Norms with a Theoretical Model.

This study was undertaken to answer the following research question: Can standardized achievement test norms be better estimated with a theoretical probability model other than the negative hypergeometric model? Ninety norms distributions from two standardized achievement test batteries (ITBS and ITED) were fitted by six models. The results strongly supported the Pearson Type I model (four moments) as the best-fitting curve. It was concluded that this Type I model could provide a viable alternative to the negative hypergeometric model for use in approximating distributions when moment estimates are obtained from multiple matrix sampling or item sampling procedures.

Some Theoretical and Empirical Results Related to McNemar's Test that the Population Correlation Coefficient Corrected for Attenuation Equals 1.0:

Correlation coefficients corrected for attenuation have been utilized frequently by educational and psychological investigators. One of the most important uses has been to determine whether or not two ostensibly different tests actually measure the same traits. This issue arises, for example, when the uniqueness of the behavioral categories in Blooms taxonomy is investigated, or when the overlap between tests of creativity and intelligence is considered. Since errors of measurement affect each score, the observed correlation between the scores will not equal 1.0, even if the two tests are, in fact, measuring the same abilities. The correlation coefficient corrected for attenuation estimates, under certain assumptions, the value of the correlation coefficient between error-free scores. If a corrected coefficient is near 1.0, the experimenter concludes that the two tests are measuring essentially similar abilities. Lord (1957) and McNemar (1958) have developed two somewhat different statistical procedures to test the hypothesis that the population correlation coefficient corrected for attenuation (ptt) is equal to 1.0. Lords procedure, which uses the maximum likelihood approach, is a large sample test based on the x distribution. McNemars test was derived via analysis of variance procedures; when its assumptions are met, it is appropriate for any sample size. McNemar provided several possible test statistics, each approximately distributed as a central F under a true hypothesis. He indicated that for practical applications the following statistic, which is a mean

A Comparison of Four Imputation Procedures in a Two-Variable Prediction System.

This study compared the effectiveness of four imputation procedures (mean, conditional mean, hot deck, and regression) in a two-variable prediction system. A total of 18,869 participants were included in the sample. Two data matrices containing missing values were created: one with randomly missing data and the other with nonrandomly missing data. Values were missing for only one of the two independent variables. The results of the study provide some support for the following conclusions: (a) the grand mean procedure is not an appropriate procedure for handling missing data; (b) when estimating the regression coefficient of the variable with missing data, the conditional mean procedure should be used; and (c) when the prediction of the dependent variable is of interest, the regression procedure should be used.

A Note on "Planning an Experiment in the Company of Measurement Error" by Levin and Subkoviak

Levin and Subkoviak (1977) contend that experi menters should consider the reliability of their measures when comparing the power of fixed-ef fects completely randomized designs with that of randomized block designs. The two major purposes of this note are (1) to show that under the condi tions specified by Levin and Subkoviak, it is not necessary to consider the reliabilities when compar ing these two designs and (2) to identify certain er rors in the illustrative example used by Levin and Subkoviak.

Relationship of elementary and secondary school achievement test scores to college performance

This study investigated the relationship between achievement test scores on the ITBS and ITED and academic success in college. The basic population consisted of approximately 2000 students who entered the University of Iowa as freshmen in 1977. Achievement test scores obtained in grades 4, 6, 8, 9, 10, 11, and 12 were available for a substantial portion of the population. Three measures of academic success in college were used: (1) first semester grade point average (December, 1977); (2) grade point average as of December, 1982; (3) graduation status (graduated/not graduated) as of December, 1982. Correlations between the achievement test scores and measures of college performance were similar to those reported for tests of the college entrance type. This study provides support for the predictive validity of the ITBS and ITED achievement test batteries.