Fedde Scheele

Introducing competency-based postgraduate medical education in the Netherlands

Medical boards around the world face the challenge of creating competency-based postgraduate training programs. Recent legislation requires that all postgraduate medical training programmes in The Netherlands be reformed.  In this article the Dutch Advisory Board for Postgraduate Curriculum Development shares some of their experiences with guiding the design of specialist training programs, based on the Canadian Medical Educational Directives for Specialists (CanMEDS).  All twenty-seven Dutch Medical Specialty Societies take three steps in designing a curriculum. First they divide the entire content of a specialty into logical units, so-called ‘themes’. The second step is discussing, for each theme, for which tasks trainees have to be instructed, guided, and assessed. Finally, for each task an assessment method is chosen to focus on a limited number of CanMEDS roles.  This leads to a three step training cycle: (i) based on their in-training assessment and practices, trainees will gather evidence on their development in a portfolio; (ii) this evidence stimulates the trainee and the supervisor to regularly reflect on a trainees global development regarding the CanMEDS roles as well as on the performance in specific tasks; (iii) a personal development plan structures future learning goals and strategies.  The experiences in the Netherlands are in line with international developments in postgraduate medical education and with the literature on workplace-based teaching and learning.

Who Wants Feedback? An Investigation of the Variables Influencing Residents' Feedback- Seeking Behavior in Relation to Night Shifts

Purpose The literature on feedback in clinical medical education has predominantly treated trainees as passive recipients. Past research has focused on how clinical supervisors can use feedback to improve a trainee’s performance. On the basis of research in social and organizational psychology, the authors reconceptualized residents as active seekers of feedback. They investigated what individual and situational variables influence residents’ feedback-seeking behavior on night shifts. Method Early in 2008, the authors sent obstetrics–gynecology residents in the Netherlands—both those in their first two years of graduate training and those gaining experience between undergraduate and graduate training—a questionnaire that assessed four predictor variables (learning and performance goal orientation, and instrumental and supportive leadership), two mediator variables (perceived feedback benefits and costs), and two outcome variables (frequency of feedback inquiry and monitoring). They used structural equation modeling software to test a hypothesized model of relationships between variables. Results The response rate was 76.5%. Results showed that residents who perceive more feedback benefits report a higher frequency of feedback inquiry and monitoring. More perceived feedback costs result mainly in more feedback monitoring. Residents with a higher learning goal orientation perceive more feedback benefits and fewer costs. Residents with a higher performance goal orientation perceive more feedback costs. Supportive physicians lead residents to perceive more feedback benefits and fewer costs. Conclusions This study showed that some residents actively seek feedback. Residents’ feedback-seeking behavior partially depends on attending physicians’ supervisory style. Residents’ goal orientations influence their perceptions of the benefits and costs of feedback-seeking.

Assessment of competence and progressive independence in postgraduate clinical training

Context  At present, competency‐based, outcome‐focused training is gradually replacing more traditional master–apprentice teaching in postgraduate training. This change requires a different approach to the assessment of clinical competence, especially given the decisions that must be made about the level of independence allowed to trainees.

Development and analysis of D-RECT, an instrument measuring residents’ learning climate

Background: Measurement of learning climates can serve as an indicator of a departments educational functioning. Aim: This article describes the development and psychometric qualities of an instrument to measure learning climates in postgraduate specialist training: the Dutch Residency Educational Climate Test (D-RECT). Method: A preliminary questionnaire was evaluated in a modified Delphi procedure. Simultaneously, all residents in the Netherlands were invited to fill out the preliminary questionnaire. We used exploratory factor analysis to analyze the outcomes and construct the definitive D-RECT. Confirmatory factor analysis tested the questionnaires goodness of fit. Generalizability studies tested the number of residents needed for a reliable outcome. Results: In two rounds, the Delphi panel reached consensus. In addition, 1278 residents representing 26 specialties completed the questionnaire. The Delphi panels input in combination with the exploratory factor analysis of 600 completed surveys led to the definitive D-RECT, consisting of 50 items and 11 subscales (e.g., feedback, supervision, patient handover and professional relations between attendings). Confirmatory factor analyses of the remaining surveys confirmed the construct. The results showed that a feasible number of residents is needed for a reliable outcome. Conclusion: D-RECT appears to be a valid, reliable and feasible tool to measure the quality of clinical learning climates.

How undergraduate clinical learning climates differ: a multi-method case study

Context  The clinical learning climate affects undergraduate medical students’ behaviour, satisfaction and success. Most studies predominantly describe aspects of the clinical learning climate using quantitative methodologies, such as questionnaires. This study aimed to illuminate medical students’ perceptions of the clinical learning climate, and which factors and their interactions explain differences in clinical learning climates.

What is wrong with assessment in postgraduate training? Lessons from clinical practice and educational research

Workplace-based assessment is more commonly given a lukewarm than a warm welcome by its prospective users. In this article, we summarise the workplace-based assessment literature as well as our own experiences with workplace-based assessment to derive lessons that can facilitate acceptance of workplace-based assessment in postgraduate specialty training. We propose to shift the emphasis in workplace-based assessment from assessment of trainee performance to the learning of trainees. Workplace-based assessment should focus on supporting supervisors in taking entrustment decisions by complementing their “gut feeling” with information from assessments and focus less on assessment and testability. One of the most stubborn problems with workplace-based assessment is the absence of observation of trainees and the lack of feedback based on observations. Non-standardised observations are used to organise feedback. To make these assessments meaningful for learning, it is essential that they are not perceived as summative by their users, that they provide narrative feedback for the learner and that there is a form of facilitation that helps to integrate the feedback in trainees’ self-assessments.

Training Physicians to Provide High-Value, Cost-Conscious Care: A Systematic Review

IMPORTANCE Increasing health care expenditures are taxing the sustainability of the health care system. Physicians should be prepared to deliver high-value, cost-conscious care. OBJECTIVE To understand the circumstances in which the delivery of high-value, cost-conscious care is learned, with a goal of informing development of effective educational interventions. DATA SOURCES PubMed, EMBASE, ERIC, and Cochrane databases were searched from inception until September 5, 2015, to identify learners and cost-related topics. STUDY SELECTION Studies were included on the basis of topic relevance, implementation of intervention, evaluation of intervention, educational components in intervention, and appropriate target group. There was no restriction on study design. DATA EXTRACTION AND SYNTHESIS Data extraction was guided by a merged and modified version of a Best Evidence in Medical Education abstraction form and a Cochrane data coding sheet. Articles were analyzed using the realist review method, a narrative review technique that focuses on understanding the underlying mechanisms in interventions. Recurrent patterns were identified in the data through thematic analyses. Resulting themes were discussed within the research team until consensus was reached. MAIN OUTCOMES AND MEASURES Main outcomes were factors that promote education in delivering high-value, cost-conscious care. FINDINGS The initial search identified 2650 articles; 79 met the inclusion criteria, of which 14 were randomized clinical trials. The majority of the studies were conducted in North America (78.5%) using a pre-post interventional design (58.2%; at least 1619 participants); they focused on practicing physicians (36.7%; at least 3448 participants), resident physicians (6.3%; n = 516), and medical students (15.2%; n = 275). Among the 14 randomized clinical trials, 12 addressed knowledge transmission, 7 reflective practice, and 1 supportive environment; 10 (71%) concluded that the intervention was effective. The data analysis suggested that 3 factors aid successful learning: (1) effective transmission of knowledge, related, for example, to general health economics and prices of health services, to scientific evidence regarding guidelines and the benefits and harms of health care, and to patient preferences and personal values (67 articles); (2) facilitation of reflective practice, such as providing feedback or asking reflective questions regarding decisions related to laboratory ordering or prescribing to give trainees insight into their past and current behavior (56 articles); and (3) creation of a supportive environment in which the organization of the health care system, the presence of role models of delivering high-value, cost-conscious care, and a culture of high-value, cost-conscious care reinforce the desired training goals (27 articles). CONCLUSIONS AND RELEVANCE Research on educating physicians to deliver high-value, cost-conscious care suggests that learning by practicing physicians, resident physicians, and medical students is promoted by combining specific knowledge transmission, reflective practice, and a supportive environment. These factors should be considered when educational interventions are being developed.

Experiences with EPAs, potential benefits and pitfalls

Abstract Reforms in postgraduate medical education (PGME) exposed a gap between educational theory and clinical practice. Entrustable Professional Activities (EPAs) were introduced to assist clinicians in bridging this gap and to create better consonance between the intended and the enacted curriculum. In this viewpoint paper, we discuss the potential and the pitfalls of using EPAs in PGME. EPAs promise an effective way of teaching abstract competencies in a curriculum based on real-life professional activities that are suitable for clinical assessment. Summative judgement is used to entrust a resident step by step in a certain EPA, resulting in an increase of independent practice. However, we argue that the success of EPAs depends on (1) a balance: brief focussed descriptions against the requirements for detail and (2) a precondition: a mature and flexible workplace for learning.

A qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education

Introduction: Recent changes in postgraduate medical training curricula usually encompass a shift towards more formative assessment, or assessment for learning. However, though theoretically well suited to postgraduate training, evidence is emerging that engaging in formative assessment in daily clinical practice is complex. Aim: We aimed to explore trainees’ and supervisors’ perceptions of what factors determine active engagement in formative assessment. Methods: Focus group study with postgraduate trainees and supervisors in obstetrics and gynaecology. Results: Three higher order themes emerged: individual perspectives on feedback, supportiveness of the learning environment and the credibility of feedback and/or feedback giver. Conclusion: Engaging in formative assessment with a genuine impact on learning is complex and quite a challenge to both trainees and supervisors. Individual perspectives on feedback, a supportive learning environment and credibility of feedback are all important in this process. Every one of these should be taken into account when the utility of formative assessment in postgraduate medical training is evaluated.

Mentoring portfolio use in undergraduate and postgraduate medical education

Aim: Mentoring is widely acknowledged as being crucial for portfolio learning. The aim of this study is to examine how mentoring portfolio use has been implemented in undergraduate and postgraduate settings. Method: The results of interviews with six key persons involved in setting up portfolio use in medical education programmes were used to develop a questionnaire, which was administered to 30 coordinators of undergraduate and postgraduate portfolio programmes in the Netherlands and Flanders. Results: The interviews yielded four main aspects of the portfolio mentoring process – educational aims, individual meetings, small group sessions and mentor characteristics. Based on the questionnaire data, 16 undergraduate and 14 postgraduate programmes were described. Providing feedback and stimulating reflection were the main objectives of the mentoring process. Individual meetings were the favourite method for mentoring (26 programmes). Small group sessions to support the use of portfolios were held in 16 programmes, mostly in the undergraduate setting. In general, portfolio mentors were clinically qualified academic staff trained for their mentoring tasks. Conclusion: This study provides a variety of practical insights into implementing mentoring processes in portfolio programmes.