Harold G. J. Bok

Curriculum development for the workplace using Entrustable Professional Activities (EPAs) : AMEE Guide No. 99

Abstract This Guide was written to support educators interested in building a competency-based workplace curriculum. It aims to provide an up-to-date overview of the literature on Entrustable Professional Activities (EPAs), supplemented with suggestions for practical application to curriculum construction, assessment and educational technology. The Guide first introduces concepts and definitions related to EPAs and then guidance for their identification, elaboration and validation, while clarifying common misunderstandings about EPAs. A matrix-mapping approach of combining EPAs with competencies is discussed, and related to existing concepts such as competency milestones. A specific section is devoted to entrustment decision-making as an inextricable part of working with EPAs. In using EPAs, assessment in the workplace is translated to entrustment decision-making for designated levels of permitted autonomy, ranging from acting under full supervision to providing supervision to a junior learner. A final section is devoted to the use of technology, including mobile devices and electronic portfolios to support feedback to trainees about their progress and to support entrustment decision-making by programme directors or clinical teams.

Programmatic assessment of competency-based workplace learning: when theory meets practice

BackgroundIn competency-based medical education emphasis has shifted towards outcomes, capabilities, and learner-centeredness. Together with a focus on sustained evidence of professional competence this calls for new methods of teaching and assessment. Recently, medical educators advocated the use of a holistic, programmatic approach towards assessment. Besides maximum facilitation of learning it should improve the validity and reliability of measurements and documentation of competence development. We explored how, in a competency-based curriculum, current theories on programmatic assessment interacted with educational practice.MethodsIn a development study including evaluation, we investigated the implementation of a theory-based programme of assessment. Between April 2011 and May 2012 quantitative evaluation data were collected and used to guide group interviews that explored the experiences of students and clinical supervisors with the assessment programme. We coded the transcripts and emerging topics were organised into a list of lessons learned.ResultsThe programme mainly focuses on the integration of learning and assessment by motivating and supporting students to seek and accumulate feedback. The assessment instruments were aligned to cover predefined competencies to enable aggregation of information in a structured and meaningful way. Assessments that were designed as formative learning experiences were increasingly perceived as summative by students. Peer feedback was experienced as a valuable method for formative feedback. Social interaction and external guidance seemed to be of crucial importance to scaffold self-directed learning. Aggregating data from individual assessments into a holistic portfolio judgement required expertise and extensive training and supervision of judges.ConclusionsA programme of assessment with low-stakes assessments providing simultaneously formative feedback and input for summative decisions proved not easy to implement. Careful preparation and guidance of the implementation process was crucial. Assessment for learning requires meaningful feedback with each assessment. Special attention should be paid to the quality of feedback at individual assessment moments. Comprehensive attention for faculty development and training for students is essential for the successful implementation of an assessment programme.

Clarifying students' feedback-seeking behaviour in clinical clerkships

Context  Why and how do students seek feedback on their performance in the clinical workplace and which factors influence this? These questions have remained largely unanswered in research into workplace learning during clinical clerkships. Research on feedback has focused mainly on feedback providers. Whether and how feedback recipients actively seek feedback are under‐examined issues. Research in organisational psychology has proposed a mechanism whereby feedback seeking is influenced by motives and goal orientation mediated by the perceived costs and benefits of feedback. Building on a recently published model of resident doctors’ feedback‐seeking behaviour, we conducted a qualitative study to explore students’ feedback‐seeking behaviours in the clinical workplace.

Believing is seeing: how people's beliefs influence goals, emotions and behaviour

Health care professionals work and learn in complex environments. Some are able to continue learning from their practice and the challenges it presents, whereas others refrain from investing more effort when faced with setbacks. This paper discusses a social cognitive model of motivation that helps to explain the different kinds of behaviour that emerge when individuals are confronted with challenges.

Development and validation of a competency framework for veterinarians

Changing demands from society and the veterinary profession call for veterinary medical curricula that can deliver veterinarians who are able to integrate specific and generic competencies in their professional practice. This requires educational innovation directed by an integrative veterinary competency framework to guide curriculum development. Given the paucity of relevant information from the veterinary literature, a qualitative multi-method study was conducted to develop and validate such a framework. A competency framework was developed based on the analysis of focus group interviews with 54 recently graduated veterinarians and clients and subsequently validated in a Delphi procedure with a panel of 29 experts, representing the full range and diversity of the veterinary profession. The study resulted in an integrated competency framework for veterinary professionals, which consists of 16 competencies organized in seven domains: veterinary expertise, communication, collaboration, entrepreneurship, health and welfare, scholarship, and personal development. Training veterinarians who are able to use and integrate the seven domains in their professional practice is an important challenge for todays veterinary medical schools. The Veterinary Professional (VetPro) framework provides a sound empirical basis for the ongoing debate about the direction of veterinary education and curriculum development.

Feedback-giving behaviour in performance evaluations during clinical clerkships

Abstract Context: Narrative feedback documented in performance evaluations by the teacher, i.e. the clinical supervisor, is generally accepted to be essential for workplace learning. Many studies have examined factors of influence on the usage of mini-clinical evaluation exercise (mini-CEX) instruments and provision of feedback, but little is known about how these factors influence teachers’ feedback-giving behaviour. In this study, we investigated teachers’ use of mini-CEX in performance evaluations to provide narrative feedback in undergraduate clinical training. Methods: We designed an exploratory qualitative study using an interpretive approach. Focusing on the usage of mini-CEX instruments in clinical training, we conducted semi-structured interviews to explore teachers’ perceptions. Between February and June 2013, we conducted interviews with 14 clinicians participated as teachers during undergraduate clinical clerkships. Informed by concepts from the literature, we coded interview transcripts and iteratively reduced and displayed data using template analysis. Results: We identified three main themes of interrelated factors that influenced teachers’ practice with regard to mini-CEX instruments: teacher-related factors; teacher–student interaction-related factors, and teacher–context interaction-related factors. Four issues (direct observation, relationship between teacher and student, verbal versus written feedback, formative versus summative purposes) that are pertinent to workplace-based performance evaluations were presented to clarify how different factors interact with each other and influence teachers’ feedback-giving behaviour. Embedding performance observation in clinical practice and establishing trustworthy teacher–student relationships in more longitudinal clinical clerkships were considered important in creating a learning environment that supports and facilitates the feedback exchange. Conclusion: Teachers’ feedback-giving behaviour within the clinical context results from the interaction between personal, interpersonal and contextual factors. Increasing insight into how teachers use mini-CEX instruments in daily practice may offer strategies for creating a professional learning culture in which feedback giving and seeking would be enhanced.

Teachers’ perceptions of aspects affecting seminar learning: a qualitative study

BackgroundMany medical schools have embraced small group learning methods in their undergraduate curricula. Given increasing financial constraints on universities, active learning groups like seminars (with 25 students a group) are gaining popularity. To enhance the understanding of seminar learning and to determine how seminar learning can be optimised it is important to investigate stakeholders’ views. In this study, we qualitatively explored the views of teachers on aspects affecting seminar learning.MethodsTwenty-four teachers with experience in facilitating seminars in a three-year bachelor curriculum participated in semi-structured focus group interviews. Three focus groups met twice with an interval of two weeks led by one moderator. Sessions were audio taped, transcribed verbatim and independently coded by two researchers using thematic analysis. An iterative process of data reduction resulted in emerging aspects that influence seminar learning.ResultsTeachers identified seven key aspects affecting seminar learning: the seminar teacher, students, preparation, group functioning, seminar goals and content, course coherence and schedule and facilities. Important components of these aspects were: the teachers’ role in developing seminars (‘ownership’), the amount and quality of preparation materials, a non-threatening learning climate, continuity of group composition, suitability of subjects for seminar teaching, the number and quality of seminar questions, and alignment of different course activities.ConclusionsThe results of this study contribute to the unravelling of the ‘the black box’ of seminar learning. Suggestions for ways to optimise active learning in seminars are made regarding curriculum development, seminar content, quality assurance and faculty development.

Patients and learners: time for a re‐evaluation of our goals in bringing them together

assess student performance on a clinical clerkship. Teach Learn Med 2002;14:5–10. 11 Hauer KE, Cate O, Boscardin C, Irby DM, Iobst W, O’Sullivan PS. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ 2013; doi:10.1007/s10459-013-9474-4 [Epub ahead of print]. 12 Ling L, Derstine P, Cohen N. Implementing milestones and clinical competency committees. http://www.acgme-nas.org/assets/ pdf/ACGMEMilestones-CCCAssesmentWebinar.pdf. [Accessed 17 September 2013.] 13 ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med 2007;82: 542–7. 14 Thomas MR, Beckman TJ, Mauck KF, Cha SS, Thomas KG. Group assessments of resident physicians improve reliability and decrease halo error. J Gen Intern Med 2011;26:759–64. 15 Williams RG, Schwind CJ, Dunnington GL, Fortune J, Rogers D, Boehler M. Applied research: the effects of group dynamics on resident progress committee deliberations. Teach Learn Med 2005;17: 96–100. 16 Kerr NL, Tindale RS. Group performance and decision making. Annu Rev Psychol 2004;55:623–55. 17 Bornstein BH, Greene E. Jury decision making: implications for and from psychology. Curr Dir Psychol Sci 2011;20: 63–7. 18 Islam G, Zyphur MJ. Power, voice, and hierarchy: exploring the antecedents of speaking up in groups. Group Dyn Theory Res Pract 2005;9:93–103.

International survey of veterinarians to assess the importance of competencies in professional practice and education

OBJECTIVE To determine the perceived importance of specific competencies in professional veterinary practice and education among veterinarians in several countries. DESIGN Survey-based prospective study. SAMPLE 1,137 veterinarians in 10 countries. PROCEDURES Veterinarians were invited via email to participate in the study. A framework of 18 competencies grouped into 7 domains (veterinary expertise, communication, collaboration, entrepreneurship, health and welfare, scholarship, and personal development) was used. Respondents rated the importance of each competency for veterinary professional practice and for veterinary education by use of a 9-point Likert scale in an online questionnaire. Quantitative statistical analyses were performed to assess the data. RESULTS All described competencies were perceived as having importance (with overall mean ratings [all countries] ≥ 6.45/9) for professional practice and education. Competencies related to veterinary expertise had the highest ratings (overall mean, 8.33/9 for both professional practice and education). For the veterinary expertise, entrepreneurship, and scholarship domains, substantial differences (determined on the basis of statistical significance and effect size) were found in importance ratings among veterinarians in different countries. CONCLUSIONS AND CLINICAL RELEVANCE Results indicated a general consensus regarding the importance of specific types of competencies in veterinary professional practice and education. Further research into the definition of competencies essential for veterinary professionals is needed to help inform an international dialogue on the subject.

Competency-based veterinary education: an integrative approach to learning and assessment in the clinical workplace

When graduating from veterinary school, veterinary professionals must be ready to enter the complex veterinary profession. Therefore, one of the major responsibilities of any veterinary school is to develop training programmes that support students’ competency development on the trajectory from novice student to veterinary professional. The integration of learning and assessment in the clinical workplace to foster this competency development in undergraduate veterinary education was the central topic of this thesis.