Benjamin C. M. Boerebach

A Systematic Review of the Impact of Physicians’ Occupational Well-Being on the Quality of Patient Care

BackgroundIt is widely held that the occupational well-being of physicians may affect the quality of their patient care. Yet, there is still no comprehensive synthesis of the evidence on this connection.PurposeThis systematic review studied the effect of physicians’ occupational well-being on the quality of patient care.MethodsWe systematically searched PubMed, Embase, and PsychINFO from inception until August 2014. Two authors independently reviewed the studies. Empirical studies that explored the association between physicians’ occupational well-being and patient care quality were considered eligible. Data were systematically extracted on study design, participants, measurements, and findings. The Medical Education Research Study Quality Instrument (MERSQI) was used to assess study quality.ResultsUltimately, 18 studies were included. Most studies employed an observational design and were of average quality. Most studies reported positive associations of occupational well-being with patient satisfaction, patient adherence to treatment, and interpersonal aspects of patient care. Studies reported conflicting findings for occupational well-being in relation to technical aspects of patient care. One study found no association between occupational well-being and patient health outcomes.ConclusionsThe association between physicians’ occupational well-being and health care’s ultimate goal—improved patient health—remains understudied. Nonetheless, research up till date indicated that physicians’ occupational well-being can contribute to better patient satisfaction and interpersonal aspects of care. These insights may help in shaping the policies on physicians’ well-being and quality of care.

Confirmatory Factor Analysis of the System for Evaluation of Teaching Qualities (SETQ) in Graduate Medical Training

The System for Evaluation of Teaching Qualities (SETQ) was developed as a formative system for the continuous evaluation and development of physicians’ teaching performance in graduate medical training. It has been seven years since the introduction and initial exploratory psychometric analysis of the SETQ questionnaires. This study investigates the validity and reliability of the SETQ questionnaires across hospitals and medical specialties using confirmatory factor analyses (CFAs), reliability analysis, and generalizability analysis. The SETQ questionnaires were tested in a sample of 3,025 physicians and 2,848 trainees in 46 hospitals. The CFA revealed acceptable fit of the data to the previously identified five-factor model. The high internal consistency estimates suggest satisfactory reliability of the subscales. These results provide robust evidence for the validity and reliability of the SETQ questionnaires for evaluating physicians’ teaching performance.

The Teacher, the Physician and the Person: Exploring Causal Connections between Teaching Performance and Role Model Types Using Directed Acyclic Graphs

Background In fledgling areas of research, evidence supporting causal assumptions is often scarce due to the small number of empirical studies conducted. In many studies it remains unclear what impact explicit and implicit causal assumptions have on the research findings; only the primary assumptions of the researchers are often presented. This is particularly true for research on the effect of faculty’s teaching performance on their role modeling. Therefore, there is a need for robust frameworks and methods for transparent formal presentation of the underlying causal assumptions used in assessing the causal effects of teaching performance on role modeling. This study explores the effects of different (plausible) causal assumptions on research outcomes. Methods This study revisits a previously published study about the influence of faculty’s teaching performance on their role modeling (as teacher-supervisor, physician and person). We drew eight directed acyclic graphs (DAGs) to visually represent different plausible causal relationships between the variables under study. These DAGs were subsequently translated into corresponding statistical models, and regression analyses were performed to estimate the associations between teaching performance and role modeling. Results The different causal models were compatible with major differences in the magnitude of the relationship between faculty’s teaching performance and their role modeling. Odds ratios for the associations between teaching performance and the three role model types ranged from 31.1 to 73.6 for the teacher-supervisor role, from 3.7 to 15.5 for the physician role, and from 2.8 to 13.8 for the person role. Conclusions Different sets of assumptions about causal relationships in role modeling research can be visually depicted using DAGs, which are then used to guide both statistical analysis and interpretation of results. Since study conclusions can be sensitive to different causal assumptions, results should be interpreted in the light of causal assumptions made in each study.

Embracing the Complexity of Valid Assessments of Clinicians' Performance: A Call for In-Depth Examination of Methodological and Statistical Contexts That Affect the Measurement of Change.

Assessments of clinicians’ professional performance have become more entrenched in clinical practice globally. Systems and tools have been developed and implemented, and factors that impact performance in response to assessments have been studied. The validity and reliability of data yielded by assessment tools have been studied extensively. However, there are important methodological and statistical issues that can impact the assessment of performance and change that are often omitted or ignored by research and practice. In this article, the authors aim to address five of these issues and show how they can impact the validity of performance and change assessments, using empirical illustrations based on longitudinal data of clinicians’ teaching performance. Specifically, the authors address the following: characteristics of a measurement scale that affect the performance data yielded by an assessment tool; different summary statistics of the same data that lead to opposing conclusions about performance and performance change; performance at the item level that does not easily translate to overall performance; how estimating performance change from two time-indexed measurements and assessing change retrospectively yield different results; and the context that affects performance and performance assessments. The authors explain how these issues affect the validity of performance assessments and offer suggestions for how to correct these issues.

Clinical teaching performance improvement of faculty in residency training: A prospective cohort study

Abstract Purpose: The purpose of this study is to investigate how aspects of a teaching performance evaluation system may affect faculty’s teaching performance improvement as perceived by residents over time. Methods: Prospective multicenter cohort study conducted in The Netherlands between 1 September 2008 and 1 February 2013. Nine hundred and one residents and 1068 faculty of 65 teaching programs in 16 hospitals were invited to annually (self-) evaluate teaching performance using the validated, specialty-specific System for Evaluation of Teaching Qualities (SETQ). We used multivariable adjusted generalized estimating equations to analyze the effects of (i) residents’ numerical feedback, (ii) narrative feedback, and (iii) faculty’s participation in self-evaluation on residents’ perception of faculty’s teaching performance improvement. Results: The average response rate over three years was 69% for faculty and 81% for residents. Higher numerical feedback scores were associated with residents rating faculty as having improved their teaching performance one year following the first measurement (regression coefficient, b: 0.077; 95% CI: 0.002–0.151; p = 0.045), but not after the second wave of receiving feedback and evaluating improvement. Receiving more suggestions for improvement was associated with improved teaching performance in subsequent years. Conclusions: Evaluation systems on clinical teaching performance appear helpful in enhancing teaching performance in residency training programs. High performing teachers also appear to improve in the perception of the residents.

Beyond limitations: Improving how we handle uncertainty in health professions education research.

Abstract The researchers’ assumptions invariably influence research outcomes. This is true for both qualitative and quantitative studies. Assumptions or choices regarding underlying theories, causal relations, study setting and population, sampling strategies, participant non-response, data collection, data analysis, and researchers’ perceptions and interpretations of results are among factors that can induce uncertainty in research findings. Researchers tend to treat these factors as potential study limitations, but how they may impact research findings is rarely explicated and, therefore, mostly unknown. In this article, we approach uncertainty as unavoidable in research and argue that communicating about uncertainty can inform researchers, policy makers and practitioners about the validity and applicability of the study findings for their interests and contexts. We illustrate approaches to address, interpret, and explicate uncertainty in medical education research in both qualitative and quantitative paradigms. Across research paradigms, we call on researchers to consider the uncertainty in their research findings, employ appropriate methods to explore its extent and effects in their work, and communicate it explicitly in their research papers. This will help to advance our understanding of the nature and implications of the emerging knowledge in our field.

Evaluating clinicians’ teaching performance

Evaluations of clinicians’ teaching performance are usually a preliminary, although essential, activity in quality management and improvement activities. This PhD project focused on testing the validity, reliability and impact of a performance evaluation system named the System of Evaluation of Teaching Qualities (SETQ) across specialities and centres in the Netherlands. The results of this project show that the SETQ questionnaires can provide clinicians with valid and reliable performance feedback that can enhance their teaching performance. Also, we tried to investigate the predictive validity of the SETQ. In conclusion, the SETQ appears to be a helpful tool for improving clinicians’ teaching performance.