Logan L. Watts

Are Ethics Training Programs Improving? A Meta-Analytic Review of Past and Present Ethics Instruction in the Sciences

Given the growing public concern and attention placed on cases of research misconduct, government agencies and research institutions have increased their efforts to develop and improve ethics education programs for scientists. The present study sought to assess the impact of these increased efforts by sampling empirical studies published since the year 2000. Studies published prior to 2000 examined in other meta-analytic work were also included to provide a baseline for assessing gains in ethics training effectiveness over time. In total, this quantitative review consisted of 66 empirical studies, 106 ethics courses, 150 effect sizes, and 10,069 training participants. Overall, the findings indicated that ethics instruction resulted in sizable benefits to participants and has improved considerably within the last decade. A number of specific findings also emerged regarding moderators of instructional effectiveness. Recommendations are discussed for improving the development, delivery, and evaluation of ethics instruction in the sciences.

Evaluating Ethics Education Programs: A Multilevel Approach

Although education in the responsible conduct of research is considered necessary, evidence bearing on the effectiveness of these programs in improving research ethics has indicated that, although some programs are successful, many fail. Accordingly, there is a need for systematic evaluation of ethics education programs. In the present effort, we examine procedures for evaluation of ethics education programs from a multilevel perspective: examining both within-program evaluation and cross-program evaluation. With regard to within-program evaluation, we note requisite designs and measures for conducting systematic program evaluation have been developed and multiple measures should be applied in program evaluation. With regard to cross-program evaluation, we argue that a meta-analytic framework should be employed where analyses are used to identify best practices in ethics education. The implications of this multilevel approach for improving responsible conduct of research educational programs are discussed.

How Do We Know What Works? A Review and Critique of Current Practices in Ethics Training Evaluation

ABSTRACT Training is a costly investment. As such, it is of great interest to know the extent to which that investment is yielding a positive return. Recent meta-analytic efforts have observed that ethics training programs are, indeed, having a positive effect, leading to the conclusion that the programs are working. However, they have also uncovered considerable variability in the effectiveness of ethics training programs, which leads to the purpose of the present study—to review current practices in ethics training evaluation. Through this review of 243 studies, consisting of 380 ethics trainings, we identified major themes in evaluation practices and training design. The tradeoffs associated with these evaluation procedures and designs are discussed, along with directions for future research.

Modeling the Instructional Effectiveness of Responsible Conduct of Research Education: A Meta-Analytic Path-Analysis

Predictive modeling in education draws on data from past courses to forecast the effectiveness of future courses. The present effort sought to identify such a model of instructional effectiveness in scientific ethics. Drawing on data from 235 courses in the responsible conduct of research (RCR), structural equation modeling techniques were used to test a predictive model of RCR course effectiveness. Fit statistics indicated the model fit the data well, with the instructional characteristics included in the model explaining approximately 85% of the variance in RCR instructional effectiveness. Implications for using the model to develop and improve future RCR courses are discussed.

A Meta-analytic Comparison of Face-to-Face and Online Delivery in Ethics Instruction: The Case for a Hybrid Approach

Despite the growing body of literature on training in the responsible conduct of research, few studies have examined the effectiveness of delivery formats used in ethics courses (i.e., face-to-face, online, hybrid). The present effort sought to address this gap in the literature through a meta-analytic review of 66 empirical studies, representing 106 ethics courses and 10,069 participants. The frequency and effectiveness of 67 instructional and process-based content areas were also assessed for each delivery format. Process-based contents were best delivered face-to-face, whereas contents delivered online were most effective when restricted to compliance-based instructional contents. Overall, hybrid courses were found to be most effective, suggesting that ethics courses are best delivered using a blend of formats and content areas. Implications and recommendations for future development of ethics education courses in the sciences are discussed.

A Comparison of the Effects of Ethics Training on International and US Students

Abstract As scientific and engineering efforts become increasingly global in nature, the need to understand differences in perceptions of research ethics issues across countries and cultures is imperative. However, investigations into the connection between nationality and ethical decision-making in the sciences have largely generated mixed results. In Study 1 of this paper, a measure of biases and compensatory strategies that could influence ethical decisions was administered. Results from this study indicated that graduate students from the United States and international graduate students studying in the US are prone to different biases. Based on these findings, recommendations are made for developing ethics education interventions to target these decision-making biases. In Study 2, we employed an ethics training intervention based on ethical sensemaking and used a well-established measure of ethical decision-making that more fully captures the content of ethical judgment. Similar to Study 1, the results obtained in this study suggest differences do exist between graduate students from the US and international graduate students in ethical decision-making prior to taking the research ethics training. However, similar effects were observed for both groups following the completion of the ethics training intervention.

Biomedical ethics education may benefit from integrating compliance and analysis approaches.

Two approaches have been widely applied in ethics education in the biomedical sciences: field-specific compliance and professional decision processes. Both approaches offer unique benefits to trainees, but future courses that integrate these two approaches might prove particularly valuable.

Effective Practices in the Delivery of Research Ethics Education: A Qualitative Review of Instructional Methods

ABSTRACT In order to delineate best practices for courses on research ethics, the goal of the present effort was to identify themes related to instructional methods reflected in effective research ethics and responsible conduct of research (RCR) courses. By utilizing a qualitative review, four themes relevant to instructional methods were identified in effective research ethics courses: active participation, case-based activities, a combination of individual and group approaches, and a small number of instructional methods. Three instructional method themes associated with less effective courses were also identified: passive learning, a group-based approach, and a large number of instructional methods. Key characteristics of each theme, along with example courses relative to each theme, are described. Additionally, implications regarding these instructional method themes and recommendations for best practices in research ethics courses are discussed.

Re-examining the Relationship Between Need for Cognition and Creativity: Predicting Creative Problem Solving Across Multiple Domains

Prior studies have demonstrated inconsistent findings with regard to the relationship between need for cognition and creativity. In our study, measurement issues were explored as a potential source of these inconsistencies. Structural equation modeling techniques were used to examine the factor structure underlying the 18-item need for cognition scale in three independent samples drawn from three prior studies. In addition, the relationships between need for cognition and the quality, originality, and elegance of creative problem solutions across multiple domains of performance were analyzed. Across all 3 samples, the bifactor model fit the data better than the 2-factor and 1-factor need for cognition models. After controlling for method-specific variance, the trait factor of the bifactor model showed consistently positive relationships with all 3 facets of creative performance and showed stronger relationships with these facets than the composite score results reported in prior studies. Implications for research and practice are discussed.

Cross-Field Comparison of Ethics Education: Golden Rules and Particulars.

ABSTRACT Research misconduct negatively impacts the scientific community and society in general. Providing training in the responsible conduct of research (RCR) to researchers is one viable approach to minimizing research misconduct. Although recent evidence suggests ethics training can indeed be effective, little empirical work has examined the similarities and differences across fields. In the present study, we analyzed 62 empirical studies in engineering, biomedical science, social science, and mixed fields. The findings suggest certain instructional principles, or “golden rules,” apply generally to all fields. These golden rules include maintaining a field-specific or field-general approach and emphasizing processes in training. The findings also suggest that content areas contributing to instructional effectiveness vary as a function of field. Generally, it appears that all fields may benefit from taking a multi-pronged approach to ethics education wherein the salient field issues are covered. Implications for RCR education are discussed.