Li Cai

Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation

Homoskedasticity is an important assumption in ordinary least squares (OLS) regression. Although the estimator of the regression parameters in OLS regression is unbiased when the homoskedasticity assumption is violated, the estimator of the covariance matrix of the parameter estimates can be biased and inconsistent under heteroskedasticity, which can produce significance tests and confidence intervals that can be liberal or conservative. After a brief description of heteroskedasticity and its effects on inference in OLS regression, we discuss a family of heteroskedasticity-consistent standard error estimators for OLS regression and argue investigators should routinely use one of these estimators when conducting hypothesis tests using OLS regression. To facilitate the adoption of this recommendation, we provide easy-to-use SPSS and SAS macros to implement the procedures discussed here.

Testing Differences between Nested Covariance Structure Models: Power Analysis and Null Hypotheses.

For comparing nested covariance structure models, the standard procedure is the likelihood ratio test of the difference in fit, where the null hypothesis is that the models fit identically in the population. A procedure for determining statistical power of this test is presented where effect size is based on a specified difference in overall fit of the models. A modification of the standard null hypothesis of zero difference in fit is proposed allowing for testing an interval hypothesis that the difference in fit between models is small, rather than zero. These developments are combined yielding a procedure for estimating power of a test of a null hypothesis of small difference in fit versus an alternative hypothesis of larger difference.

The Social Ecology of Adolescent Alcohol Misuse.

A conceptual framework based on social ecology, social learning, and social control theories guided identification of social contexts, contextual attributes, and joint effects that contribute to development of adolescent alcohol misuse. Modeling of alcohol use, suggested by social learning theory, and indicators of the social bond, suggested by social control theory, were examined in the family, peer, school, and neighborhood contexts. Interactions between alcohol modeling and social bond indicators were tested within and between contexts. Data were from a longitudinal study of 6,544 students, 1,663 of their parents, and the U.S. Census. All contexts were uniquely implicated in development of alcohol misuse from ages 11 through 17 years, and most alcohol modeling effects were contingent on attributes of social bonds.

Generalized Full-Information Item Bifactor Analysis

Full-information item bifactor analysis is an important statistical method in psychological and educational measurement. Current methods are limited to single-group analysis and inflexible in the types of item response models supported. We propose a flexible multiple-group item bifactor analysis framework that supports a variety of multidimensional item response theory models for an arbitrary mixing of dichotomous, ordinal, and nominal items. The extended item bifactor model also enables the estimation of latent variable means and variances when data from more than 1 group are present. Generalized user-defined parameter restrictions are permitted within or across groups. We derive an efficient full-information maximum marginal likelihood estimator. Our estimation method achieves substantial computational savings by extending Gibbons and Hedekers (1992) bifactor dimension reduction method so that the optimization of the marginal log-likelihood requires only 2-dimensional integration regardless of the dimensionality of the latent variables. We use simulation studies to demonstrate the flexibility and accuracy of the proposed methods. We apply the model to study cross-country differences, including differential item functioning, using data from a large international education survey on mathematics literacy.

Metropolis-Hastings Robbins-Monro Algorithm for Confirmatory Item Factor Analysis:

Item factor analysis (IFA), already well established in educational measurement, is increasingly applied to psychological measurement in research settings. However, high-dimensional confirmatory IFA remains a numerical challenge. The current research extends the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm, initially proposed for exploratory IFA, to the case of maximum likelihood estimation under user-defined linear restrictions for confirmatory IFA. MH-RM naturally integrates concepts such as the missing data formulation, data augmentation, the Metropolis algorithm, and stochastic approximation. In a limited simulation study, the accuracy of the MH-RM algorithm is checked against the standard Bock-Aitkin expectation-maximization (EM) algorithm. To demonstrate the efficiency and flexibility of the MH-RM algorithm, it is applied to the IFA of real data from pediatric quality-of-life (QOL) research in comparison with adaptive quadrature-based EM algorithm. The particular data set required a confirmatory item factor model with eight factors and a variety of equality and fixing constraints to implement the hypothesized factor pattern. MH-RM converged in less than 3 minutes to the maximum likelihood solution while the EM algorithm spent well over 4 hourrs.

Limited‐information goodness‐of‐fit testing of item response theory models for sparse 2P tables

Bartholomew and Leung proposed a limited-information goodness-of-fit test statistic (Y) for models fitted to sparse 2(P ) contingency tables. The null distribution of Y was approximated using a chi-squared distribution by matching moments. The moments were derived under the assumption that the model parameters were known in advance and it was conjectured that the approximation would also be appropriate when the parameters were to be estimated. Using maximum likelihood estimation of the two-parameter logistic item response theory model, we show that the effect of parameter estimation on the distribution of Y is too large to be ignored. Consequently, we derive the asymptotic moments of Y for maximum likelihood estimation. We show using a simulation study that when the null distribution of Y is approximated using moments that take into account the effect of estimation, Y becomes a very useful statistic to assess the overall goodness of fit of models fitted to sparse 2(P) tables.

Pooling data from multiple longitudinal studies: the role of item response theory in integrative data analysis.

There are a number of significant challenges researchers encounter when studying development over an extended period of time, including subject attrition, the changing of measurement structures across groups and developmental periods, and the need to invest substantial time and money. Integrative data analysis is an emerging set of methodologies that allows researchers to overcome many of the challenges of single-sample designs through the pooling of data drawn from multiple existing developmental studies. This approach is characterized by a host of advantages, but this also introduces several new complexities that must be addressed prior to broad adoption by developmental researchers. In this article, the authors focus on methods for fitting measurement models and creating scale scores using data drawn from multiple longitudinal studies. The authors present findings from the analysis of repeated measures of internalizing symptomatology that were pooled from three existing developmental studies. The authors describe and demonstrate each step in the analysis and conclude with a discussion of potential limitations and directions for future research.

Peer Smoking, Other Peer Attributes, and Adolescent Cigarette Smoking: A Social Network Analysis

Peer attributes other than smoking have received little attention in the research on adolescent smoking, even though the developmental literature suggests the importance of multiple dimensions of adolescent friendships and peer relations. Social network analysis was used to measure the structure of peer relations (i.e., indicators of having friends, friendship quality, and status among peers) and peer smoking (i.e., friend and school smoking). We used three-level hierarchical growth models to examine the contribution of each time-varying peer variable to individual trajectories of smoking from age 11 to 17 while controlling for the other variables, and we tested interactions between the peer structure and peer smoking variables. Data were collected over five waves of assessment from a longitudinal sample of 6,579 students in three school districts. Findings suggest a greater complexity in the peer context of smoking than previously recognized.

SEM of another flavour: Two new applications of the supplemented EM algorithm

The supplemented EM (SEM) algorithm is applied to address two goodness-of-fit testing problems in psychometrics. The first problem involves computing the information matrix for item parameters in item response theory models. This matrix is important for limited-information goodness-of-fit testing and it is also used to compute standard errors for the item parameter estimates. For the second problem, it is shown that the SEM algorithm provides a convenient computational procedure that leads to an asymptotically chi-squared goodness-of-fit statistic for the two-stage EM procedure of fitting covariance structure models in the presence of missing data. Both simulated and real data are used to illustrate the proposed procedures.

The Langer-Improved Wald Test for DIF Testing With Multiple Groups Evaluation and Comparison to Two-Group IRT

Differential item functioning (DIF) occurs when the probability of responding in a particular category to an item differs for members of different groups who are matched on the construct being measured. The identification of DIF is important for valid measurement. This research evaluates an improved version of Lord’s χ2 Wald test for comparing item response model parameter estimates between two groups. The improved version uses better approaches for computation of the covariance matrix and equating the item parameters across groups. There are two equating algorithms implemented in IRTPro and flexMIRT software: Wald-1 (one-stage) and Wald-2 (two-stage), only one of which has been studied in simulations before. The present study evaluates for the first time the Wald-1 algorithm and Wald-1 and Wald-2 for three groups simultaneously. A comparison to two-group IRT-LR-DIF is included. Results indicate that Wald-1 performs very well and is recommended, whereas Type I error is extremely inflated for Wald-2. Performance of IRT-LR-DIF and Wald-1 was similar, even for three groups.