Johnson Ching-Hong Li

Effectiveness of group cognitive-behavioral treatment for childhood anxiety in community clinics.

This study evaluated the effectiveness of cognitive-behavioral treatment for childhood anxiety in a community clinic setting in Hong Kong, China. Forty-five clinically-referred children (age 6-11 years) were randomly assigned to either a cognitive-behavioral treatment program or a waitlist-control condition. Children in the treatment condition showed significant reduction in anxiety symptoms-both statistically and clinically-whereas children in the waitlist condition did not. After the waitlist period was over, the control group also received the treatment program and showed a similar reduction in symptoms. For the full sample of 45 children, the effectiveness of the intervention was significant immediately after treatment and in 3- and 6-month follow-ups. In addition, childrens anxiety cognition and their ability to cope with anxiety-provoking situations fully mediated the treatment gains. These results offer empirical support for cognitive-behavioral treatment programs in a non-Western cultural context and plausible mediators for how cognitive-behavioral therapy works.

Psychometric properties of the Spence Children's Anxiety Scale in a Hong Kong Chinese community sample

This study evaluated the psychometric properties of a widely used self-report anxiety scale-the Spence Childrens Anxiety Scale (SCAS) and the associated parent-report version (PSCAS)-in a Hong Kong Chinese community sample. While good psychometric properties of SCAS and PSCAS had been documented in Western cultural contexts (e.g., Australia), no systematic psychometric evaluation of the Chinese-translated SCAS and PSCAS has been published. In this study, psychometric properties of SCAS and PSCAS were examined with respect to four criteria: (a) factor structure, (b) descriptive statistics, (c) convergent validity with an anxiety cognition measure, and (d) internal consistency. Psychometric properties of SCAS and PSCAS for a Chinese community sample were found to be highly comparable with those published on Australian samples, thus providing a solid conceptual foundation for use of the Chinese version of SCAS and PSCAS.

Bootstrap standard error and confidence intervals for the correlations corrected for indirect range restriction

The standard Pearson correlation coefficient, r, is a biased estimator of the population correlation coefficient, ρ(XY) , when predictor X and criterion Y are indirectly range-restricted by a third variable Z (or S). Two correction algorithms, Thorndikes (1949) Case III, and Schmidt, Oh, and Les (2006) Case IV, have been proposed to correct for the bias. However, to our knowledge, the two algorithms did not provide a procedure to estimate the associated standard error and confidence intervals. This paper suggests using the bootstrap procedure as an alternative. Two Monte Carlo simulations were conducted to systematically evaluate the empirical performance of the proposed bootstrap procedure. The results indicated that the bootstrap standard error and confidence intervals were generally accurate across simulation conditions (e.g., selection ratio, sample size). The proposed bootstrap procedure can provide a useful alternative for the estimation of the standard error and confidence intervals for the correlation corrected for indirect range restriction.

A Cross-Cultural Validation of the Learning-Related Boredom Scale (LRBS) With Canadian and Chinese College Students

This study evaluated the psychometric properties of the Learning-Related Boredom Scale (LRBS) from the Academic Emotions Questionnaire (AEQ; Pekrun, Goetz, & Perry, 2005; Pekrun, Goetz, Titz, & Perry, 2002) in a sample of 405 university students from Canada and China. Multigroup confirmatory factor analysis was used to test the factor structure and measurement invariance of the LRBS across cultural settings, after which the relationships between the LRBS, boredom frequency in class, intrinsic motivation, and self-efficacy for self-regulated learning (SESRL) were examined. Results showed evidence of reliability and measurement invariance of the LRBS, and the relationships between the LRBS, boredom frequency, and SESRL were similar across settings. The study thus provided evidence that learning-related boredom is a valid construct across culturally diverse school settings and supported the use of the LRBS in both Canadian and Chinese student populations.

Evaluating the performance of different procedures for constructing confidence intervals for coefficient alpha: a simulation study.

Reliability is one of the most important aspects of testing in educational and psychological measurement. The construction of confidence intervals for reliability coefficients has important implications for evaluating the accuracy of the sample estimate of reliability and for comparing different tests, scoring rubrics, or training procedures for raters or observers. The present simulation study evaluated and compared various parametric and non-parametric methods for constructing confidence intervals of coefficient alpha. Six factors were manipulated: number of items, number of subjects, population coefficient alpha, deviation from essentially parallel condition, item response distribution and type. The coverage and width of different confidence intervals were compared across simulation conditions.

Evaluating Person Fit for Cognitive Diagnostic Assessment

Methods evaluating person fit for cognitive diagnostic assessment are an important area of research because failing to detect misfitting responses can lead to the misinterpretation of students’ attribute profiles, which may result in faulty remediation decisions. This article aims to examine ways of detecting person misfit for cognitive diagnostic assessments. The authors first investigated whether the well-known lz statistic, developed under the framework of item response theory, can be extended for use in the context of cognitive diagnostic models. The authors also introduce a new person fit statistic, response conformity index (RCI), developed for detecting misfitting response patterns for cognitive diagnostic assessments. The authors conduct both simulation and real data studies to compare the detection rates of lz and our new statistic.

Bootstrap confidence intervals for the mean correlation corrected for Case IV range restriction: a more adequate procedure for meta-analysis.

In this study, we proposed to use the nonparametric bootstrap procedure to construct the confidence interval for the mean correlation r corrected for Case IV range restriction in meta-analysis (i.e., ; Hunter, Schmidt, & Le, 2006). A comprehensive Monte Carlo study was conducted to evaluate the accuracy of the parametric confidence interval and 3 nonparametric bootstrap confidence intervals for r(c4). Of the 4 intervals, our results showed that the bootstrap bias-corrected and accelerated percentile interval (BCaI) yielded the most accurate results across different data situations. In addition, the mean-corrected correlation r(c4) was found to be more accurate than the uncorrected estimate. Implications of the mean-corrected correlation r(c4) and BCaI in organizational studies are also discussed.

Identifying Differential Item Functioning in Multi-Stage Computer Adaptive Testing.

The purpose of this study is to evaluate the performance of CATSIB (Computer Adaptive Testing-Simultaneous Item Bias Test) for detecting differential item functioning (DIF) when items in the matching and studied subtest are administered adaptively in the context of a realistic multi-stage adaptive test (MST). MST was simulated using a 4-item module in a 7-panel administration. Three independent variables, expected to affect DIF detection rates, were manipulated: item difficulty, sample size, and balanced/unbalanced design. CATSIB met the acceptable criteria, meaning that the Type I error and power rates met 5% and 80%, respectively, for the large reference/moderate focal sample and the large reference/large focal sample conditions. These results indicate that CATSIB can be used to consistently and accurately detect DIF on an MST, but only with moderate to large samples.

Probability-of-Superiority SEM (PS-SEM)—Detecting Probability-Based Multivariate Relationships in Behavioral Research

In behavioral research, exploring bivariate relationships between variables X and Y based on the concept of probability-of-superiority (PS) has received increasing attention. Unlike the conventional, linear-based bivariate relationship (e.g., Pearsons correlation), PS defines that X and Y can be related based on their likelihood—e.g., a student who is above mean in SAT has 63% likelihood of achieving an above-mean college GPA. Despite its increasing attention, the concept of PS is restricted to a simple bivariate scenario (X-Y pair), which hinders the development and application of PS in popular multivariate modeling such as structural equation modeling (SEM). Therefore, this study addresses an empirical-based simulation study that explores the potential of detecting PS-based relationship in SEM, called PS-SEM. The simulation results showed that the proposed PS-SEM method can detect and identify PS-based when data follow PS-based relationships, thereby providing a useful method for researchers to explore PS-based SEM in their studies. Conclusions, implications, and future directions based on the findings are also discussed.

New Reliability and Validity Evidence of the Emotional Intelligence Scale

Emotional Intelligence Scale (EIS) is a popular EI measure. Yet, it has been criticized for an unclear factor structure, and its psychometric properties were mainly examined in the Western context. This study was to evaluate its psychometric properties based on 1,724 Hong Kong undergraduate students, including its (a) factor structure, (b) internal consistency, and (c) criterion validity. We compared different factor structures reported in the literature. The confirmatory factor analysis (CFA) results supported a six-factor structure, which is tallied with Salovey and Mayer’s EI conceptualization. A multigroup CFA also rendered the structure as gender invariant. The scale was internally consistent with high McDonald’s omega coefficients. Significant association between EI and grade point average (GPA) was revealed in the faculties with people-oriented studies. Furthermore, EI was correlated with social, cognitive, and self-growth outcomes and satisfaction of university experience. The study contributes to clarify the factor structure and provides new reliability and validity evidence of the EIS in the Eastern context.