Angel Centeno

A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8

Background: Preparing healthcare professionals for teaching is regarded as essential to enhancing teaching effectiveness. Although many reports describe various faculty development interventions, there is a paucity of research demonstrating their effectiveness. Objective: To synthesize the existing evidence that addresses the question: “What are the effects of faculty development interventions on the knowledge, attitudes and skills of teachers in medical education, and on the institutions in which they work?” Methods: The search, covering the period 1980–2002, included three databases (Medline, ERIC and EMBASE) and used the keywords: staff development; in-service training; medical faculty; faculty training/development; continuing medical education. Manual searches were also conducted. Articles with a focus on faculty development to improve teaching effectiveness, targeting basic and clinical scientists, were reviewed. All study designs that included outcome data beyond participant satisfaction were accepted. From an initial 2777 abstracts, 53 papers met the review criteria. Data were extracted by six coders, using the standardized BEME coding sheet, adapted for our use. Two reviewers coded each study and coding differences were resolved through discussion. Data were synthesized using Kirkpatricks four levels of educational outcomes. Findings were grouped by type of intervention and described according to levels of outcome. In addition, 8 high-quality studies were analysed in a ‘focused picture’. Results: The majority of the interventions targeted practicing clinicians. All of the reports focused on teaching improvement and the interventions included workshops, seminar series, short courses, longitudinal programs and ‘other interventions’. The study designs included 6 randomized controlled trials and 47 quasi-experimental studies, of which 31 used a pre-test–post-test design. Key points: Despite methodological limitations, the faculty development literature tends to support the following outcomes: Overall satisfaction with faculty development programs was high. Participants consistently found programs acceptable, useful and relevant to their objectives. Participants reported positive changes in attitudes toward faculty development and teaching. Participants reported increased knowledge of educational principles and gains in teaching skills. Where formal tests of knowledge were used, significant gains were shown. Changes in teaching behavior were consistently reported by participants and were also detected by students. Changes in organizational practice and student learning were not frequently investigated. However, reported changes included greater educational involvement and establishment of collegiate networks. Key features of effective faculty development contributing to effectiveness included the use of experiential learning, provision of feedback, effective peer and colleague relationships, well-designed interventions following principles of teaching and learning, and the use of a diversity of educational methods within single interventions.Methodological issues: More rigorous designs and a greater use of qualitative and mixed methods are needed to capture the complexity of the interventions. Newer methods of performance-based assessment, utilizing diverse data sources, should be explored, and reliable and valid outcome measures should be developed. The maintenance of change over time should also be considered, as should process-oriented studies comparing different faculty development strategies. Conclusions: Faculty development activities appear highly valued by participants, who also report changes in learning and behavior. Notwithstanding the methodological limitations in the literature, certain program characteristics appear to be consistently associated with effectiveness. Further research to explore these associations and document outcomes, at the individual and organizational level, is required.

Assessment for selection for the health care professions and specialty training : consensus statement and recommendations from the Ottawa 2010 Conference

Assessment for selection in medicine and the health professions should follow the same quality assurance processes as in-course assessment. The literature on selection is limited and is not strongly theoretical or conceptual. For written testing, there is evidence of the predictive validity of Medical College Admission Test (MCAT) for medical school and licensing examination performance. There is also evidence for the predictive validity of grade point average, particularly in combination with MCAT for graduate entry but little evidence about the predictive validity of school leaver scores. Interviews have not been shown to be robust selection measures. Studies of multiple mini-interviews have indicated good predictive validity and reliability. Of other measures used in selection, only the growing interest in personality testing appears to warrant future work. Widening access to medical and health professional programmes is an increasing priority and relates to the social accountability mandate of medical and health professional schools. While traditional selection measures do discriminate against various population groups, there is little evidence on the effect of non-traditional measures in widening access. Preparation and outreach programmes show most promise. In summary, the areas of consensus for assessment for selection are small in number. Recommendations for future action focus on the adoption of principles of good assessment and curriculum alignment, use of multi-method programmatic approaches, development of interdisciplinary frameworks and utilisation of sophisticated measurement models. The social accountability mandate of medical and health professional schools demands that social inclusion, workforce issues and widening of access are embedded in the principles of good assessment for selection.

A systematic review of faculty development initiatives designed to enhance teaching effectiveness: A 10-year update: BEME Guide No. 40

Abstract Background: This review, which focused on faculty development initiatives designed to improve teaching effectiveness, synthesized findings related to intervention types, study characteristics, individual and organizational outcomes, key features, and community building. Methods: This review included 111 studies (between 2002 and 2012) that met the review criteria. Findings: Overall satisfaction with faculty development programs was high. Participants reported increased confidence, enthusiasm, and awareness of effective educational practices. Gains in knowledge and skills, and self-reported changes in teaching behaviors, were frequently noted. Observed behavior changes included enhanced teaching practices, new educational initiatives, new leadership positions, and increased academic output. Organizational changes were infrequently explored. Key features included evidence-informed educational design, relevant content, experiential learning, feedback and reflection, educational projects, intentional community building, longitudinal program design, and institutional support. Conclusion: This review holds implications for practice and research. Moving forward, we should build on current success, broaden the focus beyond individual teaching effectiveness, develop programs that extend over time, promote workplace learning, foster community development, and secure institutional support. We should also embed studies in a theoretical framework, conduct more qualitative and mixed methods studies, assess behavioral and organizational change, evaluate transfer to practice, analyse key features, and explore the role of faculty development within the larger organizational context.

Innovaciones, investigación y evidencias en educación médica: la colaboración EMBE (Educación Médica Basada en la mejor Evidencia)

Aunque la educacion ha constituido siempre un componente esencial de la profesion medica, su desarrollo explicito y especifico solo se ha iniciado en las ultimas decadas. La investigacion en educacion medica esta todavia en una etapa muy inicial de su desarrollo. Las intervenciones educativas son habitualmente complejas y de caracter cualitativo; resulta, por ello, dificil, aunque indispensable, obtener evidencias cientificas que avalen tanto los diversos metodos educativos utilizados como los instrumentos de evaluacion que se emplean en los diferentes niveles de la formacion medica. A partir de la perspectiva y de las estrategias generadas por las metodologias de la medicina basada en la evidencia, ha surgido recientemente la Colaboracion EMBE como una red internacional que tiene como objetivo prioritario recoger y promover el despliegue de la mejor evidencia de la cual se pueda disponer sobre el rigor y la calidad de los actuales y de los nuevos metodos educativos y evaluativos en el ambito de la profesion medica, a fin de optimizar el uso de la extraordinaria inversion que realiza la sociedad en la formacion de los profesionales en atencion medica.

The process of learning during an examination.

As a result of this study, we concluded that examinations represent a learning opportunity that should be recognised, and that faculty members should be made aware of their potential for promoting better learning as their actions within an examination can serve to induce learning. We propose that faculty members and students should be alerted to the potential strength of the examination as a powerful teaching and learning tool.

South American investigation on professionalism: a theoretical framework

What problem was addressed? In recent years, the need to include professionalism as a core competency in medical training has regained importance, as a response to patients’ and society’s reactions against paternalistic, authoritarian and distant physicians. Three South American schools, with the shared goal to promote professionalism among their students, joined efforts to define professionalism in the region and identify, through the use of an e-portfolio, how it could be taught and assessed. The ultimate goal is to promote teaching of professionalism in other South American schools. What was tried? Building a theoretical framework to understand professionalism in medical training was an essential task for the team. The initial focus included professionalism, humanism and reflection as main concepts to address and the use of an e-portfolio as an incident-registering tool. The first task was to agree upon a definition of professionalism that would be accepted by students and teachers, the young and the elderly, but most of all would represent the South American context. In order to define the subject, interviews and focus groups were organised among faculty members, alumni and students to collect evidence about their definitions of professionalism. With this information, we agreed to use Epstein and Hundert’s definition, where professionalism is ‘The habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and the community being served’. 1. After agreeing on a definition, an e-portfolio was implemented for students to describe and reflect upon three critical incidents experienced during clinical training. Twenty students from each school participated, recording incidents at the beginning and at the end of their clinical training. After each critical incident a team member interviewed the student, asking specific questions (Why did you choose this incident? How did you feel? What did you learn? What would you have done differently?), and provided feedback. What lessons were learned? Building a framework is important for any investigation, but defining main concepts such as professionalism, humanism and reflection and implementing an e-portfolio in collaborative work, are essential. As the investigation developed, it became necessary to define common understanding about how to interpret the students’ narratives, the critical incidents and the hidden curriculum at each school. In this way, the initial theoretical framework became flexible and expanded as a result of what was revealed in students’ reflections about critical incidents. It was also important to be flexible regarding the implementation of the e-portfolio. The three schools had different approaches and experience regarding the use of this instrument. One of the schools provided feedback to the students, another used it to register the incidents and gain deeper reflections through personal interviews, and the third could not implement the e-portfolio. An important next step will be to establish clear guidelines and team training for continued use of the e-portfolio.