Gordon W. Cheung

Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance

Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which examines the change in the goodness-of-fit index (GFI) when cross-group constraints are imposed on a measurement model. Although many studies have examined the properties of GFI as indicators of overall model fit for single-group data, there have been none to date that examine how GFIs change when between-group constraints are added to a measurement model. The lack of a consensus about what constitutes significant GFI differences places limits on measurement invariance testing. We examine 20 GFIs based on the minimum fit function. A simulation under the two-group situation was used to examine changes in the GFIs (ΔGFIs) when invariance constraints were added. Based on the results, we recommend using Δcomparative fit index, ΔGamma hat, and ΔMcDonalds Noncentrality Index to evaluate measurement invariance. These three ΔGFIs are independent of both model complexity and sample size, and are not correlated with the overall fit measures. We propose critical values of these ΔGFIs that indicate measurement invariance.

Testing Mediation and Suppression Effects of Latent Variables Bootstrapping With Structural Equation Models

Because of the importance of mediation studies, researchers have been continuously searching for the best statistical test for mediation effect. The approaches that have been most commonly employed include those that use zero-order and partial correlation, hierarchical regression models, and structural equation modeling (SEM). This study extends MacKinnon and colleagues (MacKinnon, Lockwood, Hoffmann, West, & Sheets, 2002; MacKinnon, Lockwood, & Williams, 2004, MacKinnon, Warsi, & Dwyer, 1995) works by conducting a simulation that examines the distribution of mediation and suppression effects of latent variables with SEM, and the properties of confidence intervals developed from eight different methods. Results show that SEM provides unbiased estimates of mediation and suppression effects, and that the bias-corrected bootstrap confidence intervals perform best in testing for mediation and suppression effects. Steps to implement the recommended procedures with Amos are presented.

Testing Factorial Invariance across Groups: A Reconceptualization and Proposed New Method

Many cross-cultural researchers are concerned with factorial invariance; that is, with whether or not members of different cultures associate survey items, or similar measures, with similar constructs. Researchers usually test items for factorial invariance using confirmatory factor analysis (CFA). CFA, however, poses certain problems that must be dealt with. Primary among them is standardization, the process that assigns units of measurement to the constructs (latent variables). Two standardization procedures and several minor variants have been reported in the literature, but using these procedures when testing for factorial invariance can lead to inaccurate results. In this paper we review basic theory, and propose an extension of Byrne, Shavelson, and Muthgn’s (1989) procedure for identifying non-invariant items. The extended procedure solves the standardization problem by performing a systematic comparison of all pairs of factor loadings across groups. A numerical example based upon a large published data set is presented to illustrate the utility of the new procedure, particularly with regard to partial factorial invariance.

Assessing Extreme and Acquiescence Response Sets in Cross-Cultural Research Using Structural Equations Modeling:

Extreme response styles (ERS) and acquiescence response styles (ARS) may constitute important sources of cross-cultural differences on survey-type instruments. Differences in ERS and ARS, if undetected, may give rise to spurious results that do not reflect genuine differences in attitudes or perceptions. Multiple-group confirmatory factor analysis is recommended as the most effective method of testing for ERS and ARS and determining whether cultural groups can be meaningfully compared on the basis of factor (latent) means. A detailed numerical example is provided.

Estimating and Comparing Specific Mediation Effects in Complex Latent Variable Models

This teaching note starts with a demonstration of a straightforward procedure using Mplus Version 6 to produce a bias-corrected (BC) bootstrap confidence interval for testing a specific mediation effect in a complex latent variable model. The procedure is extended to constructing a BC bootstrap confidence interval for the difference between two specific mediation effects. The extended procedure not only tells whether the strengths of any direct and mediation effects or any two specific mediation effects in a latent variable model are significantly different but also provides an estimate and a confidence interval for the difference. However, the Mplus procedures do not allow the estimation of a BC bootstrap confidence interval for the difference between two standardized mediation effects. This teaching note thus demonstrates the LISREL procedures for constructing BC confidence intervals for specific standardized mediation effects and for comparing two standardized mediation effects. Finally, procedures combining Mplus and PRELIS are demonstrated for constructing BC bootstrap confidence intervals for the difference between the between-part and within-part path coefficients in multilevel models and for examining models with interactions of latent variables.

Testing Measurement Models for Factorial Invariance: A Systematic Approach.

Comparing different groups (e.g., cultures, age cohorts) using survey-type instruments raises the question of factorial invariance, that is, whether or not members of different groups ascribe the same meanings to survey items. This article attempts to advance multi-group research by (a) providing a concise summary of the factorial invariance problem, (b) proposing a simplified notation intended to facilitate discussion of the problem, and (c) suggesting a structured approach for testing large models. This procedure is illustrated using an extended example. Two computer programs designed to make the recommended procedures less laborious are offered.

Testing Equivalence in the Structure, Means, and Variances of Higher-Order Constructs With Structural Equation Modeling

Recent developments in the literature provide us with a better understanding of the importance of and procedures for testing measurement equivalence in organizational research. However, whereas many constructs in organizational research are multidimensional and multifaceted, procedures for testing the measurement equivalence of higher-order constructs are not commonly known. This study first discusses the importance of examining the measurement equivalence of higher-order constructs from a theoretical perspective, and then demonstrates that importance with a small-scale simulation. Finally, the procedures for testing various forms of measurement equivalence with structural equation modeling for both firstand second-order constructs are clarified and illustrated with a numerical example.

The Effects of Model Parsimony and Sampling Error on the Fit of Structural Equation Models

The fit between a structural equation model and a data set is operationalized as the value of goodness-of-fit indices. The discrepancy between the estimated value and the value indicating perfect fit has three sources: misspecification, error arising from theoretical parsimony in the description of the model (parsimony error), and sampling error. Misspecification, which represents a disparity between “realworld” relationships and relationships in the model, is the most important source of error for researchers. It cannot be accurately assessed, however, unless parsimony error and sampling error are taken into account. Parsimony error occurs in measurement models when secondary relationships are excluded. Secondary relationships are defined here as secondary factor loadings and error term correlations that have small values, no theoretical bases, and no substantive meaning. A simulation was conducted to examine the effects of parsimony error on perfect measurement models and to establish appropriate criteria for model fit when parsimony error is present.

A Direct Comparison Approach for Testing Measurement Invariance

Measurement equivalence/invariance (ME/I) is a condition that should be met before meaningful comparisons of survey results across groups can be made. As an alternative to the likelihood ratio test (LRT), the change in comparative fit index (ΔCFI) rules of thumb, and the modification index (MI), this teaching note demonstrates the procedures for establishing bias-corrected (BC) bootstrap confidence intervals for testing ME/I. Unlike the LRT and ΔCFI methods, which need a different model estimation per item, the BC bootstrap confidence intervals approach can examine item-level ME/I tests using a single model. This method greatly simplifies the search for an invariant item as the reference indicator in the factor-ratio test. Also demonstrated here is how the factor-ratio test and the list-and-delete method can be extended from the metric invariance test to the scalar invariance test. Finally, the BC bootstrap confidence interval procedures for comparing uniqueness variances, factor variances, factor covariances, and latent means across groups are shown.

Subcultures in Greater China: A Comparison of Managerial Values in the People's Republic of China, Hong Kong, and Taiwan

Managerial values in the three regions that form Greater China — Hong Kong, Taiwan, and the Peoples Republic of China (PRC) — were compared. It is posited that in addition to Confucian philosophy, political and economic systems also have significant effects on the values of Chinese managers. Results show that despite the economic integration in Greater China, managerial values have yet to be unified. Managers in Hong Kong, Taiwan, and the PRC are convergent in collectivism and uncertainty avoidance. On the other hand, managers in the PRC demonstrated higher power distance and less concern about deadlines and plans than in managers Hong Kong and Taiwan. Materialism is also greater in the PRC and Hong Kong than in Taiwan.