Mark H. C. Lai

Examining the Rule of Thumb of Not Using Multilevel Modeling: The “Design Effect Smaller Than Two” Rule

Educational researchers commonly use the rule of thumb of “design effect smaller than 2” as the justification of not accounting for the multilevel or clustered structure in their data. The rule, however, has not yet been systematically studied in previous research. In the present study, we generated data from three different models (which differ in the location of the clustering effect). With a 3 (design effect) × 5 (cluster size) × 4 (number of clusters) Monte Carlo simulation study we found that the rule should not be applied when researchers: (a) are interested in the effects of higher-level predictors, or (b) have a cluster size less than 10. Implications of the findings and limitations of the study are discussed.

On the Psychometric Study of Human Life History Strategies: State of the Science and Evidence of Two Independent Dimensions

This article attends to recent discussions of validity in psychometric research on human life history strategy (LHS), provides a constructive critique of the extant literature, and describes strategies for improving construct validity. To place the psychometric study of human LHS on more solid ground, our review indicates that researchers should (a) use approaches to psychometric modeling that are consistent with their philosophies of measurement, (b) confirm the dimensionality of life history indicators, and (c) establish measurement invariance for at least a subset of indicators. Because we see confirming the dimensionality of life history indicators as the next step toward placing the psychometrics of human LHS on more solid ground, we use nationally representative data and structural equation modeling to test the structure of middle adult life history indicators. We found statistically independent mating competition and Super-K dimensions and the effects of parental harshness and childhood unpredictability on Super-K were consistent with past research. However, childhood socioeconomic status had a moderate positive effect on mating competition and no effect on Super-K, while unpredictability did not predict mating competition. We conclude that human LHS is more complex than previously suggested—there does not seem to be a single dimension of human LHS among Western adults and the effects of environmental components seem to vary between mating competition and Super-K.

Trajectories of life satisfaction five years after medical discharge for traumatically acquired disability.

OBJECTIVES We studied the predictive impact of family satisfaction, marital status, and functional impairment on the trajectories of life satisfaction over the first 5 years following medical treatment for traumatic spinal cord injury, burns, or interarticular fractures (total N = 662). It was anticipated that fewer functional impairments, being married, and greater family satisfaction would predict higher life satisfaction trajectories. METHOD The Functional Independence Measure, the Family Satisfaction Scale, and the Life Satisfaction Index were administered 12, 24, 48, and 60 months postdischarge. RESULTS Trajectory modeling revealed that greater functional impairment significantly predicted lower life satisfaction, regardless of injury type. However, this association diminished when marital status and family satisfaction were entered into the models. Greater family satisfaction and being married predicted greater life satisfaction across time. Moreover, there was no evidence for increases in life satisfaction trajectories over time: Trajectories were stable across time for all injury groups. CONCLUSIONS Results suggest that being married and greater family satisfaction promote life satisfaction among those who traumatically acquire disability, and these beneficial effects may be more salient than the degree of functional impairment imposed by the condition.

Standardized Mean Differences in Two-Level Cross-Classified Random Effects Models

Multilevel modeling techniques are becoming more popular in handling data with multilevel structure in educational and behavioral research. Recently, researchers have paid more attention to cross-classified data structure that naturally arises in educational settings. However, unlike traditional single-level research, methodological studies about multilevel effect size have been rare and those that have recently appeared had an emphasis on strictly hierarchical data structure. This article extends the work on multilevel standardized mean differences from strictly hierarchical structure to both fully and partially cross-classified structures. Analytically derived formulae for calculating effect sizes and the corresponding sampling variances (or standard errors) are presented, verified by simulation results, and illustrated with real data examples. Implications for primary research studies and meta-analyses are discussed.

Specification Search for Identifying the Correct Mean Trajectory in Polynomial Latent Growth Models.

This study investigated the optimal strategy for model specification search under the latent growth modeling (LGM) framework, specifically on searching for the correct polynomial mean or average growth model when there is no a priori hypothesized model in the absence of theory. In this simulation study, the effectiveness of different starting models on the search of the true mean growth model was investigated in terms of the mean and within-subject variance-covariance (V-C) structure model. The results showed that specifying the most complex (i.e., unstructured) within-subject V-C structure with the use of LRT, ΔAIC, and ΔBIC achieved the highest recovery rate (>85%) of the true mean trajectory. Implications of the findings and limitations are discussed.

Test Review: Advanced Clinical Solutions for WAIS-IV and WMS-IV:

The Advanced Clinical Solutions (ACS) for the Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV; Wechsler, 2008) and the Wechsler Memory Scale–Fourth Edition (WMS-IV; Wechsler, 2009) was published by Pearson in 2009. It is a clinical tool for extending the assessment of individuals’ cognitive functioning. Generally, the ACS provides supplemental information for the results of the WAIS-IV and WMS-IV; it includes six components that are relatively independent, namely, additional scores, effort measures, demographically adjusted norms, reliable change scores, test of premorbid functioning, and social cognition subtests. These new elements were specially designed for forensic evaluations, readministrations, and neuropsychological evaluations. The age range for the ACS varies for different elements, but for most tests it is 16 to 90, as consistent with the WAIS-IV and WMS-IV. A technician or graduate assistant with appropriate graduate-level training can administer and score the tests under supervision, but as stressed in the Administration and Scoring Manual results should only be interpreted by professionals with extensive training in assessment. In addition, to use the ACS for assessing neuropsychological functioning, examiners must have adequate background in neuropsychological assessment.

Motives for online gaming questionnaire: Its psychometric properties and correlation with Internet gaming disorder symptoms among Chinese people

Background and aims Internet gaming disorder (IGD) imposes a potential public health threat worldwide. Gaming motives are potentially salient factors of IGD, but research on Chinese gaming motives is scarce. This study empirically evaluated the psychometric properties of the Chinese version of the Motives for Online Gaming Questionnaire (C-MOGQ), the first inventory that measures seven different gaming motives applicable to all type of online games. We also investigated the associations between various gaming motives and IGD symptoms among Chinese gamers. Methods Three hundred and eighty-three Chinese adult online gamers (Mean age = 23.7 years) voluntarily completed our online, anonymous survey in December 2015. Results The confirmatory factor analysis results supported a bi-factor model with a general factor subsuming all C-MOGQ items (General Motivation) and seven uncorrelated domain-specific factors (Escape, Coping, Fantasy, Skill Development, Recreation, Competition, and Social). High internal consistencies of the overall scale and subscales were observed. The criterion-related validity of this Chinese version was also supported by the positive correlations of C-MOGQ scale scores with psychological need satisfaction and time spent gaming. Furthermore, we found that high General Motivation (coupled with high Escape motive and low Skill Development motive) was associated with more IGD symptoms reported by our Chinese participants. Discussion and conclusions Our findings demonstrated the utility of C-MOGQ in measuring gaming motives of Chinese online gamers, and we recommend the consideration of both its total score and subscale scores in future studies.

Coaching Behaviors, Satisfaction of Needs, and Intrinsic Motivation Among Chinese University Athletes

This cross-sectional study applied self-determination theory to understand the relationship between coaching behaviors, psychological need satisfaction, and intrinsic motivation among young Chinese athletes from 2 universities in Macao, China (N = 208). In the path model with the demographics adjusted, coaching behaviors (training instruction, democratic behaviors, autocratic behaviors, social support, and positive feedback) and need satisfaction (perceived autonomy, competence, and relatedness) explained 44% of the variance in intrinsic motivation. We also found indirect effects of some coaching behaviors on intrinsic motivation through satisfying the psychological needs. Furthermore, independent self-construal significantly moderated the relationship between perceived relatedness and intrinsic motivation.

A Modified Comparative Fit Index for Factorial Invariance Studies

As a prerequisite for meaningful comparison of latent variables across multiple populations, measurement invariance or specifically factorial invariance has often been evaluated in social science research. Alongside with the changes in the model chi-square values, the comparative fit index (CFI; Bentler, 1990) is a widely used fit index for evaluating different stages of factorial invariance, including metric invariance (equal factor loadings), scalar invariance (equal intercepts), and strict invariance (equal unique factor variances). Although previous literature generally showed that the CFI performed well for single-group structural equation modeling analyses, its applicability to multiple group analyses such as factorial invariance studies has not been examined. In this study we argue that the commonly used default baseline model for the CFI might not be suitable for factorial invariance studies because (a) it is not nested within the scalar invariance model, and thus (b) the resulting CFI values might not be sensitive to the group differences in the measurement model. We therefore proposed a modified version of the CFI with an alternative (and less restrictive) baseline model that allows observed variables to be correlated. Monte Carlo simulation studies were conducted to evaluate the utility of this modified CFI across various conditions including varying degree of noninvariance and different factorial invariance models. Results showed that the modified CFI outperformed both the conventional CFI and the ΔCFI (Cheung & Rensvold, 2002) in terms of sensitivity to small and medium noninvariance.

Testing Factorial Invariance With Unbalanced Samples

In testing the factorial invariance of a measure across groups, the groups are often of different sizes. Large imbalances in group size might affect the results of factorial invariance studies and lead to incorrect conclusions of invariance because the fit function in multiple-group factor analysis includes a weighting by group sample size. The implication is that violations of invariance might not be detected if the sample sizes of the 2 groups are severely unbalanced. In this study, we examined the effects of group size differences on results of factorial invariance tests, proposed a subsampling method to address unbalanced sample size issue in factorial invariance studies, and evaluated the proposed approach in various simulation conditions. Our findings confirm that violations of invariance might be masked in the case of severely unbalanced group size conditions and support the use of the proposed subsampling method to obtain accurate results for invariance studies.