Ángel M. Fidalgo

MHDIF: A Computer Program for Detecting Uniform and Nonuniform Differential Functioning With the Mantel-Haenszel Procedure:

MHDIF implements the Mantel-Haenszel (MH) technique proposed by Holland & Thayer (1988) for detecting differential item functioning (DIF) for each dichotomously scored item in a test. The MH ~2 statistic for testing the null hypothesis of no DIF and the h4H delta (MH D-DIF) measure of amount of DIF are calculated by MHDIF. For detecting uniform DIF, MHDIF computes the MH statistics in the standard manner (i.e., in the total sample). For detecting nonuniform DIF, MHDIF uses the modification of the MH procedure proposed by Mazor, Clauser, & Hambleton (1994). The modification consists of calculating the MH statistics separately for low-performing groups (examinees with a total score less than or equal to the mean of the test score distribution in the total sample) and high-performing groups (examinees with a total score greater than the mean of the test score distribution in the sample). For detecting uniform and nonuniform DIF, the >4Fi statistics are computed in two stages. In the first stage, the MH statistics are computed for each item. If none of the items is detected as DIF, the program stops. If any item is identified as having DIF, the test score for each examinee is refined by removing items that were found to display DIF in the first stage. In the second stage, the MH statistics are recomputed for each item, matching examinees according to their purified test score. When an item is being analyzed for DIF, it is included in the matching criterion even though it had displayed DIF in the initial analysis. The program output includes descriptive statistics on the test data, and the following values for each item: ( 1 ) the common odds ratio alpha, (2) the MH ~-~~t~, (3) the MH X2, (4) whether the item was identified as a DIF item at

Generalized Mantel-Haenszel Methods for Differential Item Functioning Detection.

Mantel-Haenszel methods comprise a highly flexible methodology for assessing the degree of association between two categorical variables, whether they are nominal or ordinal, while controlling for other variables. The versatility of Mantel-Haenszel analytical approaches has made them very popular in the assessment of the differential functioning of both dichotomous and polytomous items. Up to now, researchers have limited the use of Mantel-Haenszel statistics to analyzing contingency tables of dimensions 2 × 2 (by means of the Mantel-Haenszel chi-square statistic) and of dimensions of 2 × C (by means of either the generalized Mantel-Haenszel test or Mantels test). The main objective of this article is to illustrate a unified framework for the analysis of differential item functioning using the Mantel-Haenszel methods. This is done by means of the generalized Mantel-Haenszel statistic for the analysis of the general case of Q contingency tables with dimensions R × C. Moreover, with the new formulation in consideration, this article reviews the most recent research on differential item functioning and suggests new applications and research lines in relation to the statistics proposed.

Empirical Bayes versus Standard Mantel-Haenszel Statistics for Detecting Differential Item Functioning under Small Sample Conditions

In this study, the authors assess several strategies created on the basis of the Mantel-Haenszel (MH) procedure for conducting differential item functioning (DIF) analysis with small samples. One of the analytical strategies is a loss function (LF) that uses empirical Bayes Mantel-Haenszel estimators, whereas the other strategies use the classical MH statistics (the MH chisquare statistic using high levels of significance [0.20] or empirical criteria on the basis of the magnitude of the MH-delta estimator). The authors conducted a series of computer simulations in which they manipulated different types of tests, sample sizes, and ability distributions. The results show that the loss function does not offer advantages, in terms of power and Type I error rate, over other analytical strategies that use the classical MH statistics.

Using Generalized Mantel-Haenszel Statistics to Assess DIF Among Multiple Groups

In spite of the growing interest in cross-cultural research and assessment, there is little research on statistical procedures that can be used to simultaneously assess the differential item functioning (DIF) across multiple groups. The chief objective of this work is to show a unified framework for the analysis of DIF in multiple groups using one of the most popular methodologies for DIF assessment: the Mantel-Haenszel (MH) methods. The MH statistics proposed to date with this purpose only permitted analysis of the DIF for dichotomous items. In contrast, the statistics presented here permit, through a single significance test, simultaneous evaluation of the DIF in several groups, being applicable to both dichotomous and polytomous items. Specific software to detect DIF using this methodology is available free of charge.

Liberal and Conservative Differential Item Functioning Detection Using Mantel-Haenszel and SIBTEST: Implications for Type I and Type II Error Rates

The aim of this work was to determine, in terms of Type I and Type II error rates, the risks of applying various statistical procedures for evaluating differential item functioning. To this end, the authors carried out a simulation study in which the Mantel-Haenszel and SIBTEST procedures were applied in conjunction. The variables manipulated were sample size and distribution of ability between groups. Results indicated that, although there was a high rate of agreement between the procedures, the joint Type I and Type II error rate may vary substantially from that obtained when each of the procedures was applied separately. Furthermore, the authors analyzed empirical data to obtain information complementary to the Monte Carlo study. Two strategies that minimize each type of error and some issues of a practical nature are discussed.

INFLUENCE OF THE SUPPRESSION OF SELF-DISCREPANT THOUGHTS ON THE VIVIDNESS OF PERCEP TION OF AUDITORY ILLUSIONS

Based on the relationship between cognitive intrusions and auditory hallucinations established by Morrison and Baker (2000) and Morrison, Haddock and Tarrier (1995) the present study examines the possible effect of the repeated suppression of self-discrepant thoughts on the vividness of auditory illusions in a sample from a non-clinical population. Sixty-one participants were randomly assigned to a suppression of thoughts group (n = 31) or a focalization of thoughts group (n = 30) with different levels of self-discrepancy. After carrying out the task over a period of 48 hours, participants were presented with non-vocal auditory stimulation and asked to state whether they heard any verbalizations, and if so, how clearly. Results show how the repeated suppression of self-discrepant thoughts has a considerable effect on the vividness of illusions (F(1, 50) = 16.09; p < 0.001). The implications of these results for psychological therapy are analysed, with special emphasis on the importance of a research line based on acceptance.

GMHDIF: A Computer Program for Detecting DIF in Dichotomous and Polytomous Items Using Generalized Mantel-Haenszel Statistics

Mantel-Haenszel (MH) methods constitute one of the most popular nonparametric differential item functioning (DIF) detection procedures. To date, the statistical software designed for assessing DIF with MH procedures has used the MH chi-square statistic, wMH (Mantel & Haenszel, 1959), the generalized MH test (GMH; Mantel & Haenszel, 1959), and the Mantel test (Mantel, 1963). These statistics limit the DIF analysis to two groups. However, as described in Fidalgo and Madeira (2008), we can apply the generalized MH statistic proposed by Landis, Heyman, and Koch (1978) to DIF assessment in multiple groups, both for dichotomous items and polytomous items (Fidalgo & Scalon, 2010; Fidalgo, Quintanilla, Fernández, Pons, & Aguerri, 2010). As is pointed out there, this statistic subsumes the wMH statistic, the GHM test, and the Mantel test.

Strategies for Testing Statistical and Practical Significance in Detecting DIF with Logistic Regression Models.

This study examines three controversial aspects in differential item functioning (DIF) detection by logistic regression (LR) models: first, the relative effectiveness of different analytical strategies for detecting DIF; second, the suitability of the Wald statistic for determining the statistical significance of the parameters of interest; and third, the degree of equivalence between the main DIF classification systems. Different strategies to tests–LR models, and different DIF classification systems, were compared using data obtained from the University of Tehran English Proficiency Test (UTEPT). The data obtained from 400 test takers who hold a master’s degree in science and engineering or humanities were investigated for DIF. The data were also analyzed with the Mantel–Haenszel procedure in order to have an appropriate comparison for detecting uniform DIF. The article provides some guidelines for DIF detection using LR models that can be useful for practitioners in the field of language testing and assessment.

Influence of Metacognitive Variables and Thought Suppression on Number of Thoughts, Discomfort they Produce and Number and Quality of Auditory Illusions

Based on the model proposed by Morrison, Haddock & Tarrier (1995) on auditory hallucinations, this study explores the relationships between certain metacognitive variables and number of thoughts, the discomfort they produce, number of auditory illusions and the quality with which they are perceived in a sample from a non-clinical population. After group administration of the Metacognitions Questionnaire, 61 participants were randomly assigned to a suppression group (n = 31) or a focalization group (n = 30) in relation to thoughts with different degrees of self-discrepancy. Forty-eight hours after the set task, a non-vocal auditory stimulus was presented, and subjects were required to say whether they heard any words and, if so, how clearly. The results show how the metacognitive factors studied are useful for predicting our findings only for the suppression group and not for that of focalization. These data are discussed in the light of Morrison et al.s model of auditory hallucinations.

A Comparison between Some Generalized Mantel-Haenszel Statistics for Detecting DIF in Data Simulated under the Graded Response Model.

The main objective of this study was to establish the relative efficacy of the generalized Mantel-Haenszel test (GMH) and the Mantel test for detecting large numbers of differential item functioning (DIF) patterns. To this end this study considered a topic not dealt with in the literature to date: the possible differential effect of type of scores assigned to item-response categories on the power and Type I error rate of the Mantel test. For this purpose, a simulation study with data generated under the graded response model was carried out. The results showed that (a) the scoring system used to compute the Mantel test influences its power for detecting DIF and (b) for conditions comparable to those simulated, the GMH may be the best option, given that it is capable of detecting more complex patterns of association than the Mantel test, that is, more types of DIF.