Haiyan Bai

A Multilevel Approach to Assessing the Interaction Effects on College Student Retention

The present study utilizes a multilevel approach to assess the effects of four different types of intervention on college student retention, focusing on the interaction effects between the student characteristics and the types of intervention. The program effects on a 3-year trend are also explored. The findings of the present study reveal that the social integration programs improved the first-year retention rates for female students, the advising programs and the social integration programs worked better in the first year for students from more selective colleges within the university, and the first-year experience programs had a significant lasting effect across the 3 years on retention for elder students and male students. It is also found that the advising programs were significantly more effective on the first-year retention rates than the general orientation programs. This study provides empirical evidence for researchers and administrators in higher education to improve the effectiveness of intervention programs for students with specific characteristics.

A Multivariate Approach to a Meta-Analytic Review of the Effectiveness of the D.A.R.E. Program

The Drug Abuse Resistance Education (D.A.R.E.) program is a widespread but controversial school-based drug prevention program in the United States as well as in many other countries. The present multivariate meta-analysis reviewed 20 studies that assessed the effectiveness of the D.A.R.E. program in the United States. The results showed that the effects of the D.A.R.E. program on drug use did not vary across the studies with a less than small overall effect while the effects on psychosocial behavior varied with still a less than small overall effect. In addition, the characteristics of the studies significantly explained the variation of the heterogeneous effects on psychosocial behavior, which provides empirical evidence for improving the school-based drug prevention program.

Propensity score interval matching: using bootstrap confidence intervals for accommodating estimation errors of propensity scores.

BackgroundPropensity score methods have become a popular tool for reducing selection bias in making causal inference from observational studies in medical research. Propensity score matching, a key component of propensity score methods, normally matches units based on the distance between point estimates of the propensity scores. The problem with this technique is that it is difficult to establish a sensible criterion to evaluate the closeness of matched units without knowing estimation errors of the propensity scores.MethodsThe present study introduces interval matching using bootstrap confidence intervals for accommodating estimation errors of propensity scores. In interval matching, if the confidence interval of a unit in the treatment group overlaps with that of one or more units in the comparison group, they are considered as matched units.ResultsThe procedure of interval matching is illustrated in an empirical example using a real-life dataset from the Nursing Home Compare, a national survey conducted by the Centers for Medicare and Medicaid Services. The empirical example provided promising evidence that interval matching reduced more selection bias than did commonly used matching methods including the rival method, caliper matching. Interval matching’s approach methodologically sounds more meaningful than its competing matching methods because interval matching develop a more “scientific” criterion for matching units using confidence intervals.ConclusionsInterval matching is a promisingly better alternative tool for reducing selection bias in making causal inference from observational studies, especially useful in secondary data analysis on national databases such as the Centers for Medicare and Medicaid Services data.

A comparison of propensity score matching methods for reducing selection bias

Propensity score matching (PSM) has become a popular approach for research studies when randomization is infeasible. However, there are significant differences in the effectiveness of selection bias reduction among the existing PSM methods and, therefore, it is challenging for researchers to select an appropriate matching method. This current study compares four commonly used PSM methods for reducing selection bias on observational data from which the treatment effects are intended to be assessed. The selection bias, standardized bias and percent bias reduction are evaluated for each of the PSM methods using empirical data drawn from the national Education Longitudinal Study of 2002. The results of the current study provide empirical evidence and helpful information for researchers to select effective PSM methods for their research studies.

Resampling Methods Revisited: Advancing the Understanding and Applications in Educational Research.

Resampling methods including randomization test, cross‐validation, the jackknife and the bootstrap are widely employed in the research areas of natural science, engineering and medicine, but they lack appreciation in educational research. The purpose of the present review is to revisit and highlight the key principles and developments of resampling methods to advance the understanding and applications of resampling methods in educational research. The present review also intends to provide resources for educational researchers to identify specific resampling methods that may be unitized to solve research problems in education. The procedures of each resampling technique are illustrated by empirical examples. The recent developments and further directions of resampling methods are also discussed.

Propensity Score Methods in Nursing Research: Take Advantage of Them but Proceed With Caution.

Intervention research on health outcomes is important to advancing the nursing science. Using randomized controlled trials (RCTs) to estimate intervention (or treatment) effects is optimal for this purpose. Unfortunately, for practical or ethical reasons, RCTs are often not feasible, and thus, researchers often rely on observational or non-RCT data to estimate treatment effects. Such practice poses a threat to the validity of treatment effect estimation, because the lack of randomization introduces selection bias in observational and non-RCT data. Since the inception of Rosenbaum and Rubin’s (1983) seminal work on propensity score methods for reducing selection bias, numerous studies have been published in the social, behavioral, and medical literature in applying propensity score methods to observational studies and non-RCTs. A propensity score is defined as the conditional probability of a subject (e.g., patient) being assigned to a treatment group given a set of observed covariates (or confounders) such as age, gender, race/ethnicity, and health status (Rosenbaum & Rubin, 1985). Propensity score methods are a set of statistical procedures for using propensity scores to balance the distributions of observed covariates between the treatment and control (or comparison) groups with the aim of reducing selection bias. Such covariate balance enables a direct comparison between the treatment and control groups, similar to RCTs; therefore, applying propensity score methods can increase the validity of treatment effect estimation when observational or non-RCT data are used (Bai, 2011b; Pan & Bai, 2015b; Rubin, 2008). Propensity score methods usually consist of four basic steps: (a) selecting covariates; (b) estimating propensity scores; (c) matching, weighting, or stratifying subjects on/using propensity scores; and (d) conducting the intended outcome analysis (Pan & Bai, 2015a). Although propensity score methods are widely used in social, behavioral, and medical research, few publications related to propensity score methods are found in nursing research. For example, a search in Web of Science using “propensity score” as a topic keyword showed that, of the 14,561 articles related to propensity scores published since 1983 in Web of Science categorized journals, only 48 of them (0.33%) were published in nursing journals. The dearth of applications of propensity score methods to nursing research suggests that propensity score methods may not be well known to nursing

Application of a new resampling method to SEM: a comparison of S-SMART with the bootstrap

Among the commonly used resampling methods of dealing with small-sample problems, the bootstrap enjoys the widest applications because it often outperforms its counterparts. However, the bootstrap still has limitations when its operations are contemplated. Therefore, the purpose of this study is to examine an alternative, new resampling method (called S-SMART) and compare the statistical performance of it with that of the bootstrap through an application of them to the most advanced modelling technique, SEM, as an example. The evaluation of the statistical performances of S-SMART and the bootstrap with respect to the standard errors of the parameter estimates was conducted through a Monte Carlo simulation study. This work, while potentially benefiting educational and behavioural research, conceivably would also provide methodological support for other research areas, such as bioinformatics, biology, geosciences, astronomy, and ecology, where large samples are hard to obtain.