Oleksandr S. Chernyshenko

Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy.

In this article, the authors developed a common strategy for identifying differential item functioning (DIF) items that can be implemented in both the mean and covariance structures method (MACS) and item response theory (IRT). They proposed examining the loadings (discrimination) and the intercept (location) parameters simultaneously using the likelihood ratio test with a free-baseline model and Bonferroni corrected critical p values. They compared the relative efficacy of this approach with alternative implementations for various types and amounts of DIF, sample sizes, numbers of response categories, and amounts of impact (latent mean differences). Results indicated that the proposed strategy was considerably more effective than an alternative approach involving a constrained-baseline model. Both MACS and IRT performed similarly well in the majority of experimental conditions. As expected, MACS performed slightly worse in dichotomous conditions but better than IRT in polytomous cases where sample sizes were small. Also, contrary to popular belief, MACS performed well in conditions where DIF was simulated on item thresholds (item means), and its accuracy was not affected by impact.

A lexical investigation of the lower-order structure of conscientiousness

A principal components analysis of lexically derived trait adjectives was performed to investigate the lower-order factor structure of conscientiousness (N=1675). A solution with eight substantive components fit the data best and showed good reliability, and convergent and discriminant validity. The eight components were labeled reliability, orderliness, impulse control, decisiveness, punctuality, formalness, conventionality, and industriousness. The relevance of the structure to previous lexical research and existing personality inventories is discussed.

Constructing Personality Scales Under the Assumptions of an Ideal Point Response Process: Toward Increasing the Flexibility of Personality Measures

The main aim of this article is to explicate why a transition to ideal point methods of scale construction is needed to advance the field of personality assessment. The study empirically demonstrated the substantive benefits of ideal point methodology as compared with the dominance framework underlying traditional methods of scale construction. Specifically, using a large, heterogeneous pool of order items, the authors constructed scales using traditional classical test theory, dominance item response theory (IRT), and ideal point IRT methods. The merits of each method were examined in terms of item pool utilization, model-data fit, measurement precision, and construct and criterion-related validity. Results show that adoption of the ideal point approach provided a more flexible platform for creating future personality measures, and this transition did not adversely affect the validity of personality test scores.

Examining the effects of differential item (functioning and differential) test functioning on selection decisions: when are statistically significant effects practically important?

Item response theory differential test functioning (DTP) methods are often used to address issues in personnel selection, but the results are frequently difficult to interpret because statistically significant findings may have little practical importance. In this article, the authors proposed 2 effect size measures for DTP. One related DTP to mean raw score differences across groups: the other related DTP to the 4/5th rule for adverse impact at successive cut scores. The effects of DTP were examined in the context of personality assessment, professional licensure, and college admissions. Overall, the result indicated that although many items exhibited bias in analyses of the large samples, the net magnitudes of effect on potential selection decisions were nugatory.

Job satisfaction as mediator: An assessment of job satisfaction's position within the nomological network

Job satisfactions position within the nomological network and the mechanism outlined by theories of social exchange suggest that job satisfaction functions as a mediator of the relationship between various antecedent variables and volitional workplace behaviours. We extend social exchange theory to include perceptions of the total job situation and develop a model that positions job satisfaction as a mediator of the relationships between various internal and external antecedent variables, and three volitional workplace behaviours: citizenship behaviours, counterproductive workplace behaviours, and job withdrawal. The fit of a fully mediated model is good and all four classes of antecedents (dispositions, workplace events, job characteristics, job opportunities) contributed uniquely to the prediction of satisfaction. Job satisfaction is also shown to mediate most antecedent-consequence relationships, although two important exceptions are evident. A direct link from pro-social disposition to OCBs, and a direct link and one from anti-social disposition to counterproductivity, suggest that job satisfaction does not fully moderate the relationships between dispositions and contextual behaviours.

An IRT Approach to Constructing and Scoring Pairwise Preference Items Involving Stimuli on Different Dimensions: The Multi-Unidimensional Pairwise-Preference Model

This article proposes an item response theory (IRT) approach to constructing and scoring multidimensional pairwise preference items. Individual statements are administered and calibrated using a unidimensional single-stimulus model. Tests are created by combining multidimensional items with a small number of unidimensional pairings needed to identify the latent metric. Trait scores are then obtained using a multidimensional Bayes modal estimation procedure based on a mathematical model called MUPP, which is illustrated and tested here using Monte Carlo simulations. Simulation results show that the MUPP approach to test construction and scoring provides accurate parameter recovery in both one- and two-dimensional simulations, even with relatively few (say, 15%) unidimensional pairings. The implications of these results for constructing and scoring fake-resistant personality items are discussed.

Markov Chain Monte Carlo estimation of item parameters for the generalized graded unfolding model

The authors present a Markov Chain Monte Carlo (MCMC) parameter estimation procedure for the generalized graded unfolding model (GGUM) and compare it to the marginal maximum likelihood (MML) approach implemented in the GGUM2000 computer program, using simulated and real personality data. In the simulation study, test length, number of response options, and sample size were manipulated. Results indicate that the two methods are comparable in terms of item parameter estimation accuracy. Although the MML estimates exhibit slightly smaller bias than MCMC estimates, they also show greater variability, which results in larger root mean squared errors. Of the two methods, only MCMC provides reasonable standard error estimates for all items.

Can the Discretionary Nature of Certain Criteria Lead to Differential Prediction Across Cultural Groups

We examined the conjecture that relations between constructs across cultures may be susceptible to cultural moderation where the performance of the criterion construct is discretionary. This hypothesis was investigated using the relationship between personality and three performance constructs, with samples from the United States and New Zealand, two ideologically distinct cultures with respect to achievement orientation. All hypotheses were supported by results of hierarchical moderated regression analyses using bias free measures, suggesting that considering whether construct behaviors are discretionary is important when considering the merit of generalizing research findings across cultures.

No Evidence of a Curvilinear Relation Between Conscientiousness and Relationship, Work, and Health Outcomes

Across 2 studies and 4 samples (Ns = 8,332, 2,136, 4,963, and 753, respectively), we tested whether the relation between conscientiousness and variables associated with important aspects of individuals’ lives were curvilinear such that being high on conscientiousness was manifestly negative. Across multiple outcomes including measures of health, well-being, relationship satisfaction, job satisfaction, and organizational citizenship, we found no evidence for a systematic curvilinear relation between conscientiousness and these outcomes. Furthermore, heeding the call to use more sophisticated psychometric modeling of the conscientiousness spectrum, we used different types of scale construction and scoring methods (i.e., dominance and ideal point) and again found no evidence of curvilinear relationships between conscientiousness and the aforementioned variables. We discuss the potential reasons for the inconsistency with past research.